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PREFACE TO THE SECOND EDITION 


In revising the text of this book for a second edition, it has been my 
intention to preserve its character as an introduction to numerical 
analysis for those who want to know about numerical methods for the 
purpose of applying them in practice, when actual numbers take the 
place of the literal symbols of an algebraic formula. 

The main change from the first edition is in Chapter VII on the 
numerical integration of differential equations, which has been largely 
rearranged and somewhat extended, particularly in the treatment of 
equations with two-point boundary conditions. I have also included a 
section on Whittaker’s ‘cardinal function’ in the theory of interpola¬ 
tion, and an account of a method, due to Wilkes, of handling the 
Choleski method of matrix factorization which, in my view, converts 
this from a method for specialists (as it seemed to me when the first 
edition was written) to a practicable method for the occasional user. 
The treatment of quadrature formulae of the Gauss type has been 
extended, and a number of other sections have been added or revised. 

When the first edition was written, little had been published on pro¬ 
gramming for automatic digital computers, and it seemed advisable to 
include an introduction to this subject with some simple examples; 
these required the use of a particular programming system, which had 
to be explained. Since then, much more has been published on the sub¬ 
ject (see references on p. 284), and I have consequently excised those 
sections concerned with details of programming. 

I wish to thank various correspondents, particularly Mr. R. E. 
Beard, Mr. D. R. Bland, Mr. G. A. Erskinc, and Mr. M. Fine, who have 
written to draw my attention to points in the first edition requiring 
correction or modification. It is a pleasure to renew my thanks to the 
staff of the Clarendon Press for their co-operation. 

D. R. H. 


Cavendish Laboratory 
Cambridge 
October 1957 



















PREFACE TO THE FIRST EDITION 


This book is based on a course of lectures on Numerical Analysis which 
I have given in the Mathematical Laboratory of the University of Cam¬ 
bridge for several years. It is intended to be introductory, in the sense 
that no previous knowledge of the theory and practice of systematic 
numerical work is assumed, but it is not ‘elementary’ in the sense of 
using only school mathematics. It assumes familiarity with the calculus 
up to Taylor’s theorem and partial derivatives, acquaintance with 
differential equations and, in the chapter on linear simultaneous alge¬ 
braic equations, with some of the simpler properties of matrices. But in 
all these cases what is wanted is mainly an understanding of the ideas 
involved rather than technical facility in manipulating algebraical or 
analytical expressions. 

I have deliberately tried to restrict the algebraical and analytical 
work to the treatment of those methods which are useful in practice 
w'hen numbers are substituted for the literal symbols of a general treat¬ 
ment, and to avoid developments which are of purely formal interest. 
Such developments may be elegant mathematics, or may make the 
formal presentation more complete, but they are not contributions to 
numerical analysis, and are distracting rather than helpful to the reader 
who wants practical information about what calculations to make to 
obtain the results he requires, and how to carry them out. For a similar 
reason, I have tried to give prominence to the importance of checking in 
numerical work. Mistakes can occur in such work, and it follows that 
some process of checking is necessary to ensure that any results obtained 
are not vitiated by undetected mistakes. A treatment of numerical 
methods which does not pay some attention to this aspect of the subject 
seems to me to be quite unrealistic. In some worked examples I have 
deliberately introduced mistakes in order to show how, by suitable 
checking procedure, they can be detected, diagnosed, and corrected. 

For similar practical reasons I have deliberately omitted some 

examples of numerical work which seem to have become almost classical, 

for example the evaluation of an approximation to Jtt by application of 

1 

Gregory’s quadrature formula to the integral J dxl{\-{-x^). This par- 

0 

ticular calculation I regard as an example of how not to do numerical 
work; not because the method is wrong, but because it is not the most 
suitable for obtaining the required result (see § 6.51); use of it is therefore 
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an example of bad practice, and should not be presented as if it were an 
example of satisfactory numerical procedure. Much could be written on 
how not to do numerical calculations. I have mentioned in the text some 
procedures which should, in general, be avoided, mostly because they are 
not the most suitable for obtaining the results sought, but some because 
they are dangerous in that, if used without precautions, they may give 
wrong results. I have refrained from giving numerical illustrations of the 
dangers of such methods, except in one case (§ 5.81) in which it seemed 
advisable to give a warning, by means of a horrid example, of the dangers 
of a method for inverse interpolation which might at first sight appear 
attractive, and which has in fact been given in print as a usable method 
without any mention of the dangers. 

An introductory treatment such as that of this book cannot cover 
the subject completely; several of the chapters from Chapter IV onward 
could well be expanded to form a volume each. In particular, this book 
hardly begins to touch the needs of the specialist or research worker in 
the subject of numerical analysis; its purpose is rather to give an intro¬ 
duction to the subject to workers in other fields of pure or applied science 
who may have to carry out calculations of a non-trivial magnitude. In 
such contexts the accuracy of the approximations or measurements 
underlying the calculation to be done do not usually justify working to 
a greater accuracy than six or seven figures, and often a smaller number 
will be adequate. On the other hand, the amount of calculation to be 
done to this accuracy may be considerable. In so far as the appropriate 
method for carrying out a calculation depends on the number of figures 
kept in it, emphasis is therefore placed on methods suitable for calcula¬ 
tions of substantial extent to moderate accuracy, rather than for a few 
calculations to many figures. My personal experience of such work 
extends over 35 years, and most of this work has been concerned with 
calculations involving numerical approximations to some of the limiting 
processes of analysis, in particular integration (including the integration 
of differential equations). On the basis of this experience, I believe this 
to be one of the most important practical fields for the use of numerical 
methods, and have deliberately given it considerable prominence. 

In the past few years there has been considerable development of 
high-speed automatic general-purpose digital calculating machines. This 
development has given practical importance to the study of the process 
of organizing calculations for these machines, a process usually referred 
to as ‘programming’. This study can be regarded as a branch of 
numerical analysis, and it is one which is likely to grow in importance 
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as more of these machines become available. I have therefore included 
an introductory chapter on this subject (Chapter XII). There are 
several systems of programming, and for brevity in an introductory 
account such as this, and to avoid confusing the reader with a number 
of alternatives, it seems best to adopt one particular system in present¬ 
ing illustrative examples. I have adopted the one with which I myself 
am most familiar, and w'hich I think is one of the simplest to follow. 
But this must not be regarded as more than a means for illustrating in a 
simple form some of the general ideas involved in programming. For 
various reasons, some of which are mentioned in § 12.8, f numerical 
methods which are convenient for hand calculation with the assistance of a 
desk machine are not the most suitable for an automatic machine, and 
vice versa. But this book is intended to provide an introduction to 
numerical analysis for those who will mainly be concerned with methods 
suitable for hand calculation, and little or no reference is made to other 
methods some of which might be more suitable for automatic machines. 

In the Bibhography I have included, as suggestions for further read¬ 
ing, some books and papers not referred to in the text, but I have not 
attempted to compile a complete bibliography of numerical analysis, or 
to give references to the history of the subject; the reader who is in¬ 
terested in early references should consult The Calculus of Observations 
by Whittaker and Robinson. 

On matters concerning the use of desk calculating machines, I am 
conscious of a considerable debt to the late Dr. L. J. Comrie; the 
processes of §§ 2.25, 4.45, 4.40,andExampleTjIlearntfromhim, though 
whether he originated them I do not know; and there may well be other 
examples of his influence of which I am unconscious. Some of Dr. 
Coinrie’s long and varied experience in numerical work is incorporated 
in Chambers's Six-figure Mathematical Tables, but it is much to be 
regretted that he did not hve to write a fuller work on the art of 
numerical calculation. 

In the derivation of central-difference interpolation formulae, I have 
followed a treatment which I learnt from J. G. L. Michel, and in the 
examination of truncation errors of interpolation and integration 
formulae I have followed a treatment which I learnt from Professor 
W. E. Milne while I was serving as Acting Chief of the Institute for 
Numerical Analysis of the U.S. National Bureau of Standards. 

I wish to express my thanks to Dr. J. Howlett, of the Computing 
Section, A.E.R.E., who read the first draft typescript of this book, and 

t § 12.3 of the present edition. J Example 5 of the present ctlilion. 
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to Mr. A, S. Douglas, who read the proof sheets, for many valuable 
comments and suggestions, and for a number of corrections. Also I wish 
to thank Mr. P. Parmer for the photographs from which the drawings 
for Pigs. 1, 2, and 3 were made, Mrs. Valerie Taylor for making the 
drawings themselves, my daughter for her skill in typing much of the 
text, and Dr. M. V. Wilkes, Director of the Mathematical Laboratory, 
Cambridge, for permission to avail myself of the services of Mr. Parmer, 
Mrs. Taylor, and my daughter, all members of the staff of the Laborator}^ 
It is a pleasure also to thank those members of the staff of the Clarendon 
Press who have been concerned with the production of this book. 

D. R. H. 


Cavendish Laboratory 

Cambridge 

May 1952 
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INTRODUCTION 

1.1. What numerical analysis is about 

The subject of numerical analysis is concerned with the science and art 
of numerical calculation, and particularly with processes for getting cer¬ 
tain kinds of numerical results from certain kinds of data. The following 
are some simple typical problems for which we may require processes for 
obtaining numerical solutions; 

(i) Tabulate {smhx—x)/x^ to five decimals for x — 0(0*l)3.f 

(ii) Given such a table, 

(а) find, as accurately as possible, the value of x for which 

{smhx—x)Jx^ = 0-2; 

(б) construct a table at intervals of 0-02. 

(iii) What values of x, y, z satisfy the equations 

xyz = 6, x ^—= 6, a;-l-2y-f-3z =10? 

CO 

(iv) Tabulate | dw for x = —2(0-1)2. 

0 

(v) For what values of A has the equation 

y''-l-(A—e-®‘)y = 0 

got a solution for which y 0 as x ioo and for which 

00 

j y^dx = 1, 

— 00 

and what are the corresponding solutions ? 

Although from the point of view of numerical analysis the end to be 
attained is always a numerical result or set of results, the subject is not 
concerned with the results, that is to say answers to specific problems 
themselves, but with the.processes by which those results can be obtained. 
And although the end is a numerical result, algebra and analysis are in¬ 
volved in the develo'pment of these processes. In so far as these processes, 
and the arguments by which they are derived, are general and indepen¬ 
dent of the particular values of the numbers to which they may be 

f This is a standard notation for ‘from a: = 0 to 3 inclusive at intervals of 0-1 in x*; 
see § 2,3* 
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applied, the subject may properly be regarded as a branch of mathe¬ 
matics.f But the algebra and analysis must be aimed at providing or 
establishing practical methods of obtaining numerical results) otherwise 
it may be elegant mathematics, but is not a contribution to numerical 
analysis. 

This emphasis on practicable numerical processes requires a considerable 
change in attitude from that of ordinary algebra and analysis, to which 
the idea is quite foreign. Algebraical or analytical results which are 
formally complete answers may be almost or quite useless for numerical 
purposes. Consider, for example, the solution of a system of simultaneous 
linear algebraic equS,tions. Any textbook of algebra shows how this can 
be expressed in terms of ratios of determinants, and this result is often 
presented in a form which seems to imply that there is nothing more to 
be said on the subject. But direct evaluation of the solution in this form 
is certainly not the practical answer to the problem of finding a numerical 
solution of a set of simultaneous equations. 

As another example, consider the solution of 

^~l—2xy, y — 0 at a: = 0. (1,1) 

ax 

The standard textbook treatment gives 

y = e-a:* J e** da;, (1.2) 

0 

and regards this as a complete answer. And so, for numerical purposes, 

it is, provided one has a table of J e** dx. But in order to obtain such a 

0 

table it is much easier to reverse the process and solve the differential 
equation (1.1) by numerical methods, and then to evaluate the integral 
by using (1.2) in the form 

J e** dx = e**y, 

0 

than to evaluate J e*’ dx numerically directly. Again the formal text¬ 
book answer is of no practical use if numerical results are wanted in the 
end. As in these two examples, practical numerical considerations which 
are irrelevant to formal mathematics may require alternative methods 

t A. N. Whitehead has written {Introduction to Mathematics, p. 15): ‘Mathematics as a 
science commenced when first someone, probably a Greek, proved propositions about 
any things or some things, without specification of definite particular things.’ The 
methods of numerical analysis, as distinct from the details of their application in parti¬ 
cular cases, have that degree of generality which entities them to be considered part of 
mathematics. 
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for treating problems for which complete formal solutions may already 
be known. 

Another matter in which there is much greater emphasis in numerical 
analysis than in formal analysis is the checking of numerical work. 
Numerical results which are not reliable are of little or no value, and for 
this reason any process for obtaining them should include checking 
procedures for confirming that the alleged results are free from mistakes. 
This is considered further in § 1.3. 

As already mentioned, numerical analysis is concerned with processes. 
It is an active subject, one in which things happen in the course of 
carrying out numerical processes, and it cannot be learnt properly simply 
by reading about it, by following examples already worked, or even by 
watching examples being worked, any more than one can learn golf, 
tennis, or violin-playing by watching others play, without ever handling 
a club, racket, or violin. There is a great deal of difference between only 
thinking about processes for carrying out numerical calculations and 
actually carrying them out with numbers in the place of the algebraical 
symbols of a general treatment, and the student who wishes to get a 
feeling for the subject must work examples of the processes for himself. 
This is an essential part both of srtudy and research in the subject. Also, 
probably, he must make his own mistakes and spend time finding them 
and correcting them and their consequences before he really appreciates 
the importance of adequate checking. 

The processes of numerical analysis are necessarily finite processes. 
Ideas such as limiting processes. Dedekind sections, formal convergence, 
scarcely play any part in the numerical processes themselves, though 
they may be involved in the analytical arguments by which the numerical 
process is established. Related to this is the approximate nature of much 
of numerical analysis. In most applications of numerical analysis, almost 
no problems have answers which are rational numbers. But our system 
of representation of numbers is not suitable for numerical operations on 
irrational numbers, so that in most cases we have to be satisfied with 
approximations. And even when the answers are rational numbers, we 
shall often be content with decimal approximations to these rational 
numbers, if indeed we would not prefer them. 

It will be as well to end this section by explaining what numerical 
analysis is not. 

First, it is not necessarily concerned with the analysis of numbers 
obtained by observation in the course of some branch of experimental 
science; secondly, it is not closely related to statistics. Certainly numerical 
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analysis may be involved in the analysis of observational material, 
whether statistical or obtained by measurement, and the analysis of 
observations consisting of measurements may involve consideration 
of the statistics of errors of the measurements. But the subject itself is 
distinct from these two particular apphcations of it, just as it is distinct 
from its particular applications, for example, to the evaluation of super¬ 
sonic fluid flow or to the structures of atoms or stars. 

1.2. The main types of problems in numerical analysis 

The main kinds of operations which have to be carried out in the course 
of a numerical calculation, and for which numerical processes are re¬ 
quired, are the following: 

(a) Evaluation of formulae. 

{b) Solution of non-linear equations in one unknown. 

(c) Solution of systems of hnear simultaneous equations. 

(d) Inversion of matrices. 

(e) Determination of characteristic values and characteristic vectors 
of matrices. 

(/) Solution of systems of non-linear simultaneous equations. 

(S') Tabulation of standard functions. 

(h) Interpolation and subtabulation. 

(i) Integration and differentiation of a given function. 

(j) Smoothing. 

(k) Integration of ordinary differential equations. 

(l) Integration of partial differential equations. 

(m) Solution of integral equations. 

(n) Harmonic analysis. 

(o) Frequency analysis (periodogram analysis). 

Of these, (j), (n), and (o) are often concerned with analysis of observed data, 
which is not primarily the subject of numerical analysis as pointed out 
at the end of the previous section. 

A single calculation may involve a number of these operations. For 
example, evaluation of the solution of an ordinary differential equation 
may well involve any one or more of (a), (b), or (h) as well as the integra¬ 
tion process (k) itself. 

The subjects in this hst have been arranged more or less in order from 
less to more ‘advanced’; they will not, however, be taken in this order, 
since some of the ideas required in treating later items of this hst are also 
valuable in the earher ones. 
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1.3. Errors, mistakes, and checking 

There are three reasons for which the results of a numerical calculation 
may differ from the exact answer to the mathematical question con¬ 
cerned: 

(i) One (or more) of the formulae which are evaluated in the course 
of the work is derived by cutting off an infinite series after a finite 
number of terms; the errors introduced in this way are called 
‘truncation errors’; 

(ii) Only the more significant decimal digits of a number are retained, 
the less significant beyond a certain point being rejected: this 
process is called ‘rounding off’ and the errors introduced in this 
way are called ‘rounding errors’ or ‘rounding-off errors’; 

(iii) IVIistakes are made in carrying out the sequence of operations 
required to obtain the results sought. 

The distinction made here between an ‘error’ and a ‘mistake’ is this. 
A ‘mistake ’ is due to fallibility, either human on the part of the individual 
carrying out the calculation, or technical on the part of the mechanical 
or electrical aids used in the course of it, and is in principle avoidable. 
‘Errors’, in some degree, are unavoidable, except in some cases of cal¬ 
culations concerned entirely with integers or rational numbers; such 
calculations may occur, for example, in connexion with number theory, 
but are otherwise exceptional. ‘Truncation errors’ are unavoidable in 
any process which takes the place in numerical work of a limiting process 
of analysis; integration and differentiation are two important examples. 
‘Rounding errors’ are inevitable in division when the answer is a non¬ 
terminating decimal, and in the use of values of functions other than 
polynomials with rational coefficients, and are often incurred in multipli¬ 
cation also, since although it is possible to retain the {m-\-n) digits of the 
product of two numbers, one of m and the other of n digits, it is only 
exceptionally that all these digits are wanted; if one or both of the 
numbers being multiplied is subject to rounding error, some of the less 
significant digits in the product will be valueless anyway. 

It is necessary to check that the final results of a calculation are not 
vitiated either by errors or by mistakes, and in a substantial calculation 
it will usually be advisable to include a number of checks of intermediate 
results as well. It is often possible to estimate the magnitude of trunca¬ 
tion errors and so ensure that they are kept below a specified tolerance 
depending on the calculation and the accuracy required in the final results 
Rounding errors can often be rendered innocuous by carr 3 dng one or two. 
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or sometimes more, extra figures, known as ‘guarding figures’, in inter¬ 
mediate stages of the calculation; for example in calculating a compound 
interest table of _ (i.0325)J' 

for p = 0(1)100, to five decimals, by repeated use of the recurrence 
relation 1 ) = i •0325/(p), 


rounding errors greater than 6 in the sixth decimal can be avoided by 
keeping eight decimals in the intermediate values of f{p). A full analysis 
of the effect of rounding errors in any but a simple calculation may be 
fairly elaborate. 

Intermediate and final results of a calculation will usually be influenced 
by rounding errors at previous stages of the work, and in some cases the 
accumulated effects of rounding errors will result in checks not being 
satisfied exactly. Let y be the correct value of a quantity and y* the 
calculated value of it. Then there may be a range of values of y—y* 
which can be accepted as being results of rounding (and possibly trunca¬ 
tion) errors and not as indicating mistakes. The term ‘tolerance’ (in the 
sense in which it is used in machining work in engineering) will be used 
for this acceptable range of y — y*. For example, if a check consists of 
the equality of two numbers calculated by different processes, and the 
tolerance of each is ±2 in the last digit, a difference of 3 between them 
in this digit is within the tolerance on this difference, and can be passed. 

Anyone intending to undertake a serious piece of calculation should 
realize that adequate checking against mistakes is an essential part of any 
satisfactory numerical process. No one, and no machine, is infallible, and 
it may fairly be said that the ideal to aim at is not to avoid mistakes 
entirely, but to find all mistakes that are made, and so free the work from 
any unidentified mistakes. This of course is an ideal. It does not seem 
possible to ehminate mistakes with absolute certainty; it is always pos¬ 
sible that a mistake might be made in the check itself in such a way as 
to cancel the effect of an error it was devised to find. But with properly 
designed checking procedures and care in working, the probability of this 
should be neghgibly small. 

Provision of adequate checks is not, however, to be regarded as an 
excuse for mistakes ora, justification of carelessness in carrying out the 
details of numerical work. Location and diagnosis of a mistake, and 
correction of the mistake itself and of subsequent calculations vitiated 
by it, is often a time-consuming job, and a tiresome one at that; and 
moreover, if mistakes are too frequent, the probability of a mistake in 
a check masking a mistake which the check should detect may become 


INTRODUCTION 


7 


appreciable. Numerical work should always be done with care to avoid 
mistakes, and checks regarded as insurance against the occasional mis¬ 
takes which may occur even in careful work. 

Many calculations consist of the same group of arithmetical operations 
apphed repeatedly to different data. For example, if it were required to 
evaluate the function y defined by 

2/= + + (1.3) 

for a set of values of x, say x = —3-0(0-l )3-0, by evaluating and summing 
the separate terms of the series, the process of calculating y is the same 
for each value of x (except that for the smaller values of x more terms 
of the series are negligible and do not have to be evaluated explicitly). 
Such a systematic set of calculations is easier to check than one in which 
no step is similar to any other. A single value of the function y would be 
difficult to check adequately; a systematic set of values can be checked 
comparatively easily. In this case, for instance, a check might be based 
on the fact that y defined by (1.3) is a solution of the differential 
equation y” — \ -{-xy (for an example, see §3.3), but use of such a check 
depends on the behaviour oi y as a function of x, and is not applicable 
to a single isolated value of y. 

Mistakes in such a calculation are of two kinds, systematic (that is, the 
same mistake is made at the same point in each repetition of the sequence 
of arithmetical operations) and random. These can be illustrated from 
one method of evaluating the above series (1.3). Suppose the (?i-l-l)th 
term is evaluated by multiplying the ?ith by x^/(3?i-f l)(3?i-f2); then 

(third term) = (x^/56) X (second term), 

and the denominator here might be taken as 54 instead of 56 throughout 
the whole calculation for all values of x; this would be a systematic 
mistake. On the other hand, one too many or one too few zeros between 
the decimal point and the first significant figure might be taken in a single 
one of the terms of the series for a single value of x; this would be a 
random mistake. 

It is recognized by those with extensive experience of numerical work 
that there are two kinds of random mistake which are particularly easy 
to make. One is an interchange of adjacent digits; for example, 28575 
may be read or recorded as 25875. The other is repeating the wrong digit 
in a number in which two adjacent digits are the same; for example 36609 
may be read or written as 33609 or 36009. The error introduced by a 
mistake of the first of these kinds is always a multiple of 9 in terms of the 
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less significant of the two interchanged digits as unit; this may often help 
in locating and identifying a mistake of this kind. These are not, of course, 
the only kinds of mistakes that can be made: but if a check indicates the 
presence of a random mistake, knowledge that it is likely to be of one 
of these kinds may assist in diagnosing it. 

In the calculation of the function y defined by (1.3) by evaluation of 
the series, the calculations for different values of x are independent, so 
that a mistake in the calculation for one value of x does not affect those 
for later values of x. 

But in many calculations, such as a calculation of this same function y 
by numerical integration of the differential equation y" — satis¬ 

fied by it, a mistake at one stage vitiates all subsequent work. In such 
a case, it is important to have a current check on the work as it progresses 
rather than only an overall check carried out when the calculation is 
completed, otherwise the amount of work that has to be repeated if a 
mistake is made may become very considerable. All the time spent on 
work subsequently found to be vitiated by a mistake is just wasted, and 
a few experiences of this kind may be found severely discouraging, 
although really the moral should be simply that an adequate current 
check is needed. 

One kind of ‘check’ is so inadequate as to be almost worthless, namely, 
repetition of a calculation by the same individual that did it originally. 
It is much too easy to make the same mistake twice; and indeed it may 
be that having made a mistake once, one is conditioned to make it again 
on repeating the work. An independent repetition of the work by a second 
individual is better than no check, but should not be regarded as adequate. 
The only really satisfactory check is one which obtains or verifies a result 
by a different sequence of arithmetical operations, or a sequence involving 
different numbers, from that by which it was obtained. For example, values 
of cosh a; and sinha; interpolated from tables may be checked by use of 
the identity cosh^a; — sinh^a; = 1 (this does not check that they are not 
both interpolated for the wrong value of x, but this can probably be 
checked in some other way, depending on the rest of the calculation for 
which values of cosh a; and sinha; are wanted); and the values of y calcu¬ 
lated from the series (1.3) can be checked by use of the differential 
equation satisfied by y. 

1.4. Arrangement of work 

In most numerical work, a working sheet will be used for recording data 
and intermediate results of the calculation. A clear and orderly arrange- 
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merit of this working sheet is a great help both in avoiding mistakes and 
in locating and correcting any that do happen to be made. Numerical 
work should not be done on odd scraps of rough paper, but laid out 
systematically and in such a way as to show how the intermediate and 
final results were obtained; and the numbers entered on the work sheet 
should be written neatly and legibly. Use of ruled paper is a help in 
keeping the layout of the work neat and clear. It is advisable to use loose 
sheets rather than a book since it is rather easy to make mistakes in 
copying from one page to another of a book; with loose sheets the number 
to be copied from one sheet, and the place to which it is to be copied on 
another, can more easily be brought close together, and the copy made 
and checked more easily. 

For work of any permanent value, it is advisable to record on the 
working sheet enough explanation of the different entries, and how they 
were obtained, for the working to be followed after the lapse of a period 
of years. 

1.5. Accuracy and precision 

In contexts in which numerical work is carried out in connexion with 
scientific and technical problems, we are often concerned with the 
numerical solution of one or a set of algebraic, differential, or integral 
equations. Then it may be convenient to distinguish between the 
accuracy to which the equations, or data used in obtaining a solution of 
them, represent the real situation to which they refer, and the accuracy 
to which the results of the numerical work represent the solution of these 
equations with these data, supposed exact. The latter is sometimes 
distinguished by being called the ‘precision’ or ‘nominal accuracy’ of 
the numerical work. 

Calculations are often carried out deliberately to a nominal accuracy 
known or expected to be higher than the accuracy of the approximations 
made in deriving the equations, or higher than that of the data used in 
their solution. There are several reasons why this may be done. We may 
be interested in the differences between the results of observation and 
of calculation, whether for the purpose of assessing the accuracy to which 
the equations do give an account of the observations, or in order to 
analyse these differences so as to derive more accurate equations or data 
to use in them. Then we want to be sure that the differences between the 
results of observation and of calculation are significant, and are not 
merely consequences of the limited nominal accuracy of the calculations. 
Or we may want to determine the difference between two solutions of 
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the equations with different values of some parameters, and to obtain 
this difference we may have to calculate the separate solutions to a 
nominal accuracy higher than that of the data. Or the results may be 
only intermediate results on which some extensive interpolation, perhaps 
in two or three variables, is going to be carried out. Both for the inter¬ 
polation process and for checking purposes, it is then desirable that these 
intermediate results should be smooth and of a nominal accuracy higher 
than required in the final results. 

In a hand calculation, however, greater nominal accuracy means more 
work, more writing in recording intermediate results, more possibilities 
of mistakes, and a longer time for the calculation. It is advisable, there¬ 
fore, to watch lest the calculation is being carried to an unnecessarily 
high nominal accuracy. In this connexion a warning may be given con¬ 
cerning the use of desk machines. Since with a desk machine it is possible 
to work to eight or ten figures, there is a tendency to get into a habit 
of working with eight or ten figures when four or five would be adequate. 
This is bad practice, and a habit which the serious student of numerical 
work should avoid for his own sake. 


II 

THE TOOLS OF NUMERICAL WORK AND 
HOW TO USE THEM 

2.1. The main tools of numerical work 

Fob carrying out the numerical details of a calculation there are four 
main kinds of tools: 

(a) Desk machines. (c) Slide rule. 

(b) Tables. (d) Graph paper. 

Of these the first and second are much the most important. 

2.2. Desk machines 

A desk calculating machine is the most important single tool for 
numerical work and anyone intending to study numerical analysis 
seriously should become familiar enough with the main kinds to use 
them with facility, without more deliberate thought for the details of 
operating the machine than a good typist gives to the operation of 
individual keys of the tj^ewriter. 

There are several kinds of desk machine, some being primarily adding 
machines whereas others have facilities for multiplication; the former 
are sometimes called ‘adding machines’ as distinct from ‘calculating 
machines ’ to emphasize this feature. The latter are the more important, 
and will be considered first; adding machines are considered in §2.26. 
Of the calculating machines some are considerably different in appear¬ 
ance and operation from others, but all are broadly similar in general 
principle. All have four main components: 

(i) A setting mechanism by which a number can be set on the 
machine. 

(ii) A register in which results of additions, subtractions, and multipli¬ 
cations are accumulated; this will be called the ‘accumulator’; 
other names for it are ‘result register’ and ‘product register’. 

(iii) A counting register, sometimes called ‘multiplier register’, on 
which a count is kept of the number of additions or subtractions 
made. 

(iv) An operating handle (in hand machines) or key-operated switch 
(in electrically-driven machines). 

The setting mechanism and registers have means for setting them to zero; 
this is called ‘clearing’. 
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Three kinds of desk calculating machines are illustrated in Figs. 1, 2, 
and 3. Three different kinds of setting mechanisms are represented in 
these three machines, and this is the main reason for the difference in 
appearance between them. 

On the Brunsviga (Fig. 1), the setting mechanism consists of a series 
of levers, one for each digital position, each lever having ten positions 
corresponding to the decimal digits 0-9. On the Marchant (Fig. 2) the 
setting mechanism consists of a keyboard on which there is a set of nine 
keys, corresponding to the digits 1-9, in a column in each digital position; 
a number isr set by pressing the appropriate key in each column. On the 
Facit (Fig. 3) there is a keyboard of only ten setting keys, corresponding 
to the digits 0-9; a number is set by pressing these keys in an order 
corresponding to the order of the digits in the number, beginning with 
the most significant. 

The accumulator can be traversed relative to the setting mechanism, 
so that the least significant digital position of the adding mechanism 
corresponds to different digital positions of the accumulator. Shifting 
the accumulator one place to the right corresponds to multiplication 
by 10. 

On all these machines multiphcation is carried out by repeated addition 
and shifting. Machines which carry out multiphcation directly, by use 
of a built-in multiphcation table, have been constructed, but some 
machines carrying out multiphcation by repeated addition are now so 
fast that, for work in which numbers are supphed manuaUy to the 
machine, there is little purpose in making machines using direct 
multiphcation. 

On hand-operated machines, addition is carried out by rotating the 
handle through one turn in one direction (clockwise, looking along the 
handle towards the body of the machine) and subtraction by rotating it 
through one turn in the other direction. Most machines have a lock on 
the handle so that once a turn has been started it must be completed, 
and often have mechanical interlocks to prevent incorrect operation. 
On an electrically-driven machine the rotation is supphed by an electric 
motor instead of directly by the operator’s hand, and the motor is con¬ 
trolled by a set of key-operated switches. 

This is not the place for an account either of the internal mechanism 
or of the details of operation of different machines; the operating proce¬ 
dure is given in booklets supphed with the machine or obtainable from 
the makers or agents, but probably can best be acquired from personal 
demonstration by someone already famihar with the machine. The 



Fig. 1. Brunsviga (hand-operated). 



Fig. 2. Marchant (electrically operated). 
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following sections deal with some general points of procedure applicable 
to roost machines. 

2.21. Addition and subtraction 

Addition is carried out by setting the addend on the setting levers or 
keys and turning the handle once positively, or on an electric machine 
by pressing the -f key or by multiplying by i with the shift-control set 
to ‘non-shift’; on some machines the latter procedure is necessary when 
it is required to hold the number set, since this is cleared after addition 
when the -|- key is used. The number set up is then added to the content 
of the accumulator. The position of the decimal point needs watching 
if the number of digits after the decimal point is different in the addend 
and in the content of the accumulator. Decimal-point markers are fur¬ 
nished on all machines (their form is different on different machines); 
in single arithmetical operations it is often unnecessary to use them, but 
they are very useful in helping to keep the position of the decimal point 
correct in carrying out sequences of operations on the machine without 
writing down intermediate results, as is sometimes possible. 

Subtraction is carried out similarly to addition, except that the handle 
is turned in the opposite direction. The result of subtracting a greater 
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number from a smaller is as follows. Let a be written for a contribution 
(—a) in any digital position, so that, for example, the number 90 can be 
written 110 and the number 88 as 112. Then the negative number —23 
(for example) which is 

— 23 = —1,000+977 = —10,000+9,977 = —1,000,000+999,977 
can be written 

— 23 = T977 = T9977 = T999977, etc. (2.1) 

The number 999...99977, to the full capacity of the accumulator of the 
machine, is called the ‘complement’ of 23, or the ‘complementary form’ 
of the number — 23; it can be regarded as a representation of the number 
— 23 in the form (2.1), with the digit 1 to the left of the most significant 
digital position of the accumulator. In a number in complementary form, 
the digits to the right of the row of 9’s are the significant figures. 

Negative results appear in such a complementary form, and, in some 
machines, a carry-over from the most significant digital position of the 
accumulator is indicated by the ringing of a bell. 

Recording of negative numbers will usually be in terms of sign and 
modulus, not in their complementary form. The translation from the 
complementary form to the modulus can be done in two ways: 

(i) Translate mentally by subtracting each digit of the complement 
except the last from 9 and subtracting the last from 10. Set the 
result on the setting levers or keys, add into the accumulator, and 
verify that the content of the accumulator is now zero. This checks 
the translation and should always be done before the result is 
recorded. If the number in complementary form is wanted in the 
accumulator for further numbers to be added to it, it can be 
recovered by subtracting the number on the setting levers. 

(ii) Transfer the number in complementary form from the accumulator 
to the setting levers or keys (see §2.22), and subtract from zero. 
This wiU give some spurious 9’s on the extreme left of the accumu¬ 
lator, but it will be easy to distinguish these from the significant 
figures of the result. 

2.22. Transfer from accumulator to setting keys or levers 

In some calculations it is necessary to transfer to the setting levers or 
keys a number formed in the accumulator as the result of previous calcu¬ 
lations. For example, in the calculation of a continued product, an inter¬ 
mediate product formed in the accumulator has to be transferred to the 
setting levers or keys to be ready for multiphcation by the next factor; 
and as already mentioned in the previous section, such a transfer is a step 
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in one method of obtaining the modulus of a negative number expressed 
in complementary form. 

Some machines are provided with facilities for direct transfer from 
accumulator to setting levers or keys. In using one that is not, the 
following procedure should be followed. 

Copy on to the setting keys or levers the number to be transferred, 
subtract it from the content of the accumulator, and verify that the result is 
zero. This checks that the number has been copied correctly on to the 
setting mechanism, and this check should always be used. 

On an electric machine, the subtraction must be done in such a way 
as not to clear the keyboard after subtraction. 

2.23. Multiplication 

Multiplication is carried out by repeated addition in each digital posi¬ 
tion of the multipher, the accumulator being traversed one place right 
or left between successive digits of the multiplier. In most cases it is best 
to carry out multiplication starting with the most significant digit of the 
multiplier, as then the order of the digits is the natural one, in which it 
is easy to remember the multipher while the multiplication is being 
carried out. 

In a few machines, mainly older models, in which the mechanism for 
carrying-over in addition does not extend to the fuU capacity of the 
accumulator, this procedure will occasionaUy lead to incorrect results. 
The best way to test whether a machine has this objectionable feature 
is to subtract 1 from 0 with the accumulator in the extreme left position, 
and see if the carry-over produces 9’s right to the extreme left-hand 
digital positions of the accumulator. If not, the best way of avoiding 
trouble is not to use such a machine; but if no other is available the 
possibility of incorrect results of this cause must be kept in mind. 
In multiplication they can be avoided by starting from the least signifi¬ 
cant digit of the multiplier, but this is inconvenient as it means taking 
the digits in the opposite order to that in which they will naturally be 
remembered. 

Appreciable time can be saved in multiplication on a hand machine 
by a procedure known as ‘short-cutting’. If, as in § 2.21, a bar over a 
digit is used to represent a negative digit in that digital position only, we 
have, for example: 

183 = 223 (l.h. 12, r.h. 7) \ 

2879 = 3m_ (l.h. 26, r.h. 7) 

369175 = 43X2^ (l.h. 31, r.h. 17) 


( 2 . 2 ) 
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Multiplication by one of these numbers can be carried out by using as 
multiplier the number in the form given on the right-hand side of the 
equalities in (2.2), and using both positive and negative directions of 
turning the handle; an appreciable number of turns may be saved in this 
way; this is the process of ‘short-cutting’. The numbers of turns taken to 
carry out a multiphcation by each of the numbers in the example (2.2), in 
its form on the left-hand side and in its form on the right-hand side of the 
equality sign, are shown in brackets. ‘Short-cutting’ should be used on 
digits over 5, and on a 5 if flanked on either side by a digit over 5; for 
users of hand machines, it should become the natural way of carrying out 
multiplications; it needs a little practice at first to become proficient and 
reliable, but ease in using it is certainly worth attaining. 

In a few old models of machines, which have not carry-over (sometimes 
called ‘tens-transmission’) mechanism in the multiplier register, only the 
moduli of the individual digits are indicated (in some machines the 
negative digits are indicated in red). Such machines should be avoided, 
or, if they have to be used, short-cutting must be used with discretion 
and particular attention should be paid to checking. 

Some electric machines are fitted with means by which multiplication 
by any digit of the multiplier and the succeeding shift of the accumulator 
can be carried out by pressing one of a set of ten keys; these machines are 
so fast that short-cutting is unnecessary. On others the complete multi¬ 
plier can be set and transferred to a register, then the multiplicand set 
and the multiplication carried out automatically; in these machines the 
operator is not concerned at all with the process of multiplication by 
individual digits. 

2.24. Division 

Division can be carried out in three ways: 

First, by multiphcation by the reciprocal of the divisor. This is par¬ 
ticularly useful when the result of the division is required to be in the 
accumulator, either in order to have further numbers added to it or for 
transfer to the setting levers or keys. In the other methods of division, 
the quotient appears in the multiplier register, and no machine has 
transfer facilities from there to the setting levers or keys; this transfer 
had to be done by hand and there is no adequate means of checking it, 
whereas if a result is in the accumulator its transfer to the setting levers 
or keys can be made mechanically or checked (see § 2.22). 

Secondly, by successive subtraction, starting from the most significant 
digit. In this process, the dividend, if not already in the accumulator as 

c 
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a result of previous operations, is set and added into the accumulator 
which has previously been cleared. The multiplier register must then be 
cleared (this is automatic in the case of automatic division on some 
electrical machines); this is a step which is rather easily overlooked. The 
divisor is then set and subtracted in the most significant position until 
the remainder is less than the divisor; the accumulator is then shifted 
one place left, and the subtraction followed by a shift is repeated. The 
result appears in the multiplier register. In order to make fuU use of the 
capacity of this register, the divisor should normally be set in such a 
position on the setting levers or keys that the quotient has a non-zero 
digit in the extreme left digital position of the multiplier register. 


b/c 


Multiplier register 



a 

c 

ah/c 

b 


Setting levers or keys 


Accumulator 


Fig. 4. (Dots ♦ show decimal-point markers used as separators.) 


Thirdly, by successive addition (sometimes called division by ‘building 
up’). In this process the accumulator is cleared, the divisor set on the 
setting levers or keys, and mtdtiplied by such a number x that the result 
in the accumulator is the dividend. This multiplication is done by a 
process which is essentially one of trial, but can be quite fast, and in which 
short-cutting can be used to some extent. It is useful when the same 
divisor is used with a number of dividends, as then this divisor can be 
set up once for all and need not be disturbed to set the new dividend. It 
is sometimes also useful for forming abjc in one operation, if the number 
of digits involved is not too large. If a is set at one side of the setting 
mechanism and c at the other, and b/c is found by building up c to b, 
then the handle has been turned a number of times corresponding to 
(bjc), and a has been multiplied by that number. This arrangement is 
shown diagrammatically in Fig. 4. Another application of the process 
of division by building up is in the calculation of \afb\, where a is a 
negative number standing in the accumulator (in complementary form) 
as the result of a previous calculation. If |61 is set, and the content of the 
accumulator built up to zero, the multiplier register will read |u/6|. 
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Most electric machines are provided with facilities for automatic divi¬ 
sion; the dividend is placed in the accumulator, either by adding it in 
after clearing the accumulator, or by forming it there as a result of 
previous calculations. The divisor is then set and the ‘automatic division’ 
key pressed; the division then proceeds without further manipulation on 
the part of the operator. 

2.25. Other calculations 

With a machine of sufficient capacity, and numbers of a few digits, it 
is possible to do two calculations simultaneously, one with numbers set 
on the extreme left and the other with numbers set on the extreme right 
of the setting levers or keyboard; an example has already been given in 
the calculation of abje in one operation. The following are two other 
examples: 

(i) 2^/1 simultaneoiLsly. Set 1 on the extreme left and 

n n 

the numbers 6 ,j successively on the extreme right; for each 
multiply by the corresponding a„. Then in the accumulator ^ 

n 

is formed on the left and ^ on the right. 

n 

(ii) 2 2 simultaneously. Set on the extreme left and 

n n 

on the extreme right, multiply by a„, and repeat for; each value 
of n. Then in the accumulator ^ <^n is formed on the left and 

n 

2 6,1 on the right. Ifthe multiplier register is no^ cleared between 

n 

each multiplication, 2 <^n i® accumulated there, but this is hardly 

n 

satisfactory, as it is then impossible to check after each multiplica¬ 
tion that the right multiplier has been used. An overall check can 
be provided by setting the pairs of numbers a„, 6 „ in succession 
and multiplying each pair by the corresponding 6 „. This would 
give 2 «« 6 „ and ^ 6 ^; the latter is likely to be wanted in contexts in 

n n 

which '2 cin 2 Wanted, and the agreement of the two 

n n 

values of 2 <^n 6 ,i would check that the right multiplier values had 

n 

been used in each calculation; it does not check the setting of the 
values of in the first, or of 6 „ in the second, of the calculations. 

2.26. Adding machines 

In adding machines the position of the accumulator relative to the 
keyboard is fixed, and there is no multiplier register. Most of them have 
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keyboard setting, and many have electrical operation controlled through 
a set of keys. 

The most useful of these machines are those which make a printed 
record of each number added into the accumulator. There are two 
operations by which a total standing in the accumulator can be printed* 
If a key marked ‘total’ is operated, the total is printed and the accumu¬ 
lator is cleared) if a key marked ‘sub-total’ is operated the total is printed 
and retained in the accumulator. A particular application of the latter 

X 

operation is in the evaluation of an integral j f{w) dw as a function of 

a 

its upper limit x, by successive addition of contributions from successive 
intervals of a;. After each contribution is added, a sub-total is taken, then 
the next contribution is set and added. The printed record consists of a 
sequence of entries, alternately contributions to the integral and values 
of the integral itself. The contributions actually used by the machine can 
then be checked against the values which should have been set. 

It should be a convention in using a machine of this kind that it is left 
with the accumulator clear; but in case this has not been done, it is 
advisable, before using it, always to ensure that the accumulator is clear 
by taking a total. 

2.3. Mathematical tables 

Mathematical tablesf form a very important aid to numerical work. 
Many calculations involve the use of values of standard functions such 
as exponentials, logarithms, circular functions, Bessel functions, the 
gamma function, and though it would be possible to calculate the required 
function values from scratch as they were wanted, this would usually 
lengthen the calculation so much as to make it impracticable. In fact, if 
tables of these functions did not already exist, it would often be worth 
constructing them as a first step in the calculations for which values of 
these functions are wanted. 

The most important tables are the following: 

Comrieand Milne-Thomson's Standard 4-Figure Tables) Chambers's 
6-Figure Mathematical Tables (2 vols. 1948-9) and Chambers's 
Shorter 6-Figure Tables (1950), edited by Comrie; Barlow's Tables 
of Squares, Cubes, Reciprocals, etc., edited by Comrie; Interpolation 
and Allied Tables (H.M. Stationery Office, 1956). 

t On this general topic, see L. Fox, The Use and Construction of Mathematical Tables 
(H.M.S.O., 1966 ). 
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The first two of these include tables of circular functions for argument 
in radians, and also tables of inverse trigonometric and hyperbolic func¬ 
tions; of the two volumes of Chambers’s 6-figure tables, the second with 
so-called ‘natural ’ values is much the more useful for work with machines. 
Interpolation and Allied Tables contains a great deal of information on 
formulae and methods for interpolation and other numerical processes 
and is a very useful and inexpensive booklet. 

For functions other than the elementary fimctions, the following are 
useful: 

Dale, 5-Figure Tables of Mathematical Functions; Jahnke-Emde, 
Tables of Functions with Formulae and Graphs; British Association 
Mathematical Tables, Vols. VI, X and Part-vols. A, B. 

The amount of tabular material available in various volumes of tables 
and scattered among various journals is very considerable. The nature 
and location of most of this material published up to the end of 1944 has 
been classified and tabulated in an Index of Mathematical Tables'\ which 
is a most valuable volume and should be known to aU undertaking any 
extensive numerical work, or even small calculations involving functions 
other than the elementary fimctions, since if a function has been tabu¬ 
lated, knowledge of this fact and an adequate reference will usually avoid 
duplicating the calculation of it. 

An important source of information, particularly regarding recent or 
current work on tabulation of functions, is the journal Mathematical 
Tables and Aids to Gomputation (generally referred to as M.T.A.G.). 

In describing a table, it is convenient to have a compact notation for 
specifying the range and interval of the argument. The notation 
X = Xi(Sx)x 2 has come to be adopted as the standard abbreviated form 
for ‘for values of x from x^ to x^ inclusive at intervals 8a:’. The notation 
mD or nS is often used for a table to m decimals, or n significant figures. 

2.31. Critical tables 

Most tables give the values of the function f{x), rounded off to a certain 
number of decimals, for a sequence of equally spaced exact values of the 
argument x. Occasionally another type of table is more convenient, 
namely, one giving the range of x for which the function/(a:), rounded off 
to a certain number of decimals, has a specified value. Such a table is 
called a critical table, and is convenient for slowly varying functions, and 
also for functions which have a limited range and for which accuracy in 

t By A. Fletcher, J. C. P. Miller, and L. Rosenhead (Scientific Computing Service, Ltd., 
London 1946). A second edition is in preparation. 
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the last figure is important; in using a critical table no interpolation is 
required, and the possibihty of an error of a unit in the last figure, which 
occurs in interpolation in an ordinary table, is avoided. 

As an example, consider a table of ^x{x—l){x—l), which appears in 
a formula for non-linear interpolation, as a function of x. A portion of 
a critical table of this function to four decimals is as follows: 


X 

Jipo) 

0*1621 

-f 0*0077 

0*1691 

0*0078 

0*1777 

0*0079 

0*1897 

0*0080 

0*2334 

0*0079 

0*2462 



The values off{x) are on lines intermediate between those on which the 
values of a: stand, and the values of a; between which a value of f{x) stands 
mark the limits of the range of x for which that is the rounded value of f{x). 
These values of x are rounded values of the inverse function f~^(y) for 
values of y = /(x) halfway between the tabular values. For example, the 
above table indicates that for values of x between 0 1777 and 0 1897 the 
function/(x) has the value -f 0*0079 to four decimals; and the value of x 
for /(x) = 0*00785 is 0*1777 to four decimals. 

It is a convention in critical tables that if x has exactly the tabular 
value, the value of f{x) to be taken is that standing above the line on which 
X stands; a reminder of this convention is often given in such tables 
by the words ‘in critical cases ascend’. 

2.32. Auxiliary variables in tables 

An important aspect of mathematical tables is the use of auxiliary 
variables to simplify interpolation. This is especially important (i) in the 
neighbourhood of a singularity, where the ordinary interpolation formu¬ 
lae, applied directly to the function values, cease to be valid, (ii) for large 
values of the argument, and (iii) for oscillating functions when the table 
has to cover a large number of periods of the oscillation. Use of auxiliary 
variables may both simplify interpolation and lessen the amount of 
material which has to be calculated and printed to provide a useful table. 

The most usual step is to tabulate an auxiliary function, but in some 
cases an auxiliary independent variable may be used instead or in 
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addition. The following are examples of the tabulation of auxiliary 
functions: 

(i) log((sin x)lx} and log((tan x)jx} for small x, in place of log sin x and 
log tan X which are infinite at a: = 0 and cannot be interpolated by 
standard formulae near x = 0. 

(ii) If/(a:) is oscillatory, it may be possible to determine an ‘amplitude 
function’ A{x) and a ‘phase function’ ^{x) such that 

f{x) = ^(a:)cos{^(a;)} 

and that A{x) and vary much more slowly than f{x). Then 
A{x) and (f){x) can be tabulated at wider intervals than f{x), and 
interpolation oi A{x) and <j){x) is easier than that of f{x) itself. This 
is particularly convenient when two functions can be expressed 
as the real and imaginary parts of A{x)ex^i(f>{x)}. An important 
example is provided by the Bessel functions, for which 

A(x) = *>[{/.(x)}»+{y.(a;OT» = x>\H<;>{x)\ 
and ^(x) = tan'i[r„(x)//,(x)] = arg[i?J‘>(x)] 

form a convenient pair of auxiliary functions except for small 
values of x. 

An example of the joint use of auxiliary functions and an auxiliary 
independent variable is provided by the elliptic integral 

V 

K{k) = J (l-it2sin20)-i d0 

0 

near A: = 1. If k' = (1—k^)^ and 

J^(k) = J{^log(4/k')+J^„ 

and iCg are regular functions of k' near k = \, and a convenient tabula¬ 
tion is and against k' as argument. 

2.4. Slide rule 

A slide rule is an instrument of limited accuracy, and of limited scope 
since it cannot easily be used for addition and subtraction; but within 
its limitations it is a valuable tool of numerical work. Two contexts in 
which it is particularly useful are the following: 

(i) When a function is tabulated at intervals too large for linear inter¬ 
polation between tabular values, more elaborate interpolation 
formulae have to be used; these will be considered in Chapter V. 
In many of these, the interpolated value is expressed as the sum 
of the value which would be obtained by linear interpolation, and 
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some other terms which can be regarded as contribution to a 
‘correction’ to this value. For some or all of these contributions, 
the accuracy attainable with a slide-rule may be adequate, and 
then it is a useful tool. 

(ii) When in the solution of a linear differential equation a particular 
integral P and a complementary function G have been evaluated, 
and a small constant multiple of G, say y(7, has to be added to P 
to give a solution satisfying specified conditions, the calculation 
of yC may often be carried out to adequate accuracy on a slide-rule. 
This is a quick calculation, because after a single setting of y, all 
the values of yC can be read off without resetting. 

As well as the usual straight slide-rule with a 10-inch scale, there is 
another form, with two cursors and a single scale in the form of a helix 
on a movable cyhnder. In the Fuller slide-rule this scale is 50 feet long, 
and this enables an accuracy of 1 in 10,000 to be obtained without diffi¬ 
cult}', and 1 in 20,000 with care. Such an instrument is cheap compared 
with a desk machine and may be found very useful in work for which its 
accuracy is adequate and in circumstances in which the cost of a desk 
machine is prohibitive. With one of these slide-rules and an adding 
machine much useful numerical work can be done, especially in contexts 
involving empirical or experimentally determined functions not specified 
to more than four- or five-figure accuracy. 

2.5. Graph paper 

Graph pa,per is more generally useful as a means of presenting results 
than as a tool for obtaining them. But there are occasions when it is 
useful as a means of doing calculations, e.g. for obtaining approximate 
results which can later be refined by more accurate methods. 

Before being used for anything more than qualitative or the roughest 
of quantitative work, graph paper should be examined for uniformity 
of ruling. Paper ruled in two colours (e.g. blue for the main ruling, with 
red for every tenth line) should be examined for the registration of the 
two colours. Paper which is ruled with every fifth or tenth line tliick 
should be examined to see that the intervals between the centres of the 
lines are uniform, and not the intervals between the edges of the lines, 
a remarkable fault in some papers.f 

2.6. Other machines 

There are other aids to numerical work of various kinds, but mostly 
large or special pieces of equipment which are unlikely to be available 
f Seo Jetfreys and Jeffreys, Methods of Mathematical Physics, chap. 9. 
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to most of those for whom this book is primarily intended. The more 
important may, however, be mentioned here. 

First, there is the ‘National’ machine,f developed from an accounting 
machine. This is an adding machine with keyboard setting mechanism 
and six registers, with facilities for adding or subtracting the number 
set on the keyboard, or the number standing in any register, into any 
combination of registers. The mechanical arrangement for controlling 
these transfers is such that it can only be used effectively in calculations 
in which the same set of operations has to be repeated successively on 
different sets of numbers; but many calculations have just this character, 
and for such calculations this machine can be very valuable. 

Secondly, there are two groups of machines for carrying out arith¬ 
metical operations on numbers represented by punchings on cards, the 
‘Hollerith’ and ‘Powers-Samas’ machines. The main machines of each 
group are a ‘tabulator’ which is a multi-register adding machine with 
printing mechanism, a sorter, and a multiplying punch which can take 
a card with two numbers punched on it, and calculate and punch their 
product. The use of these machines, and the organization of calculations 
for them, is a special technique of its own,J and hardly appropriate for 
an introductory book like the present. 

Thirdly, there are various high-speed automatic calculating machines 
which can carry out, automatically, long sequences of operations once 
they have been supplied with operating instructions in a suitably coded 
form. A short account of the principles of these machines and of the 
process of organizing calculations for them is given in Chapter XII. 

f See, for example, L. J. Comrie, Journ, Roy, Slat. Soc.j Suppleynenty 3 (1936), 87, 

J See W. J. Eckert, Punched Card Methods in Scientific Computation (Columbia 
University, 1940). 
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EVALUATION OF FORMULAE 

3.1. The significance of formulae in numerical work 

The evaluation of a given formula is the simplest kind of problem 
in numerical analysis. In a sense most problems reduce to this, as the 
numerical work itself almost always consists in substituting particular 
numerical values into a process or sequence of operations which could 
be expressed in the form of a sequence of formulae to be evaluated, even 
if they are not explicitly so expressed. In most cases the real question 
of numerical analysis is. What is the best formula or set of formulae to 
evaluate in order to obtain the required result ?, and it is with this ques¬ 
tion that we shall primarily be concerned in later chapters. But equally 
important questions for practical work are how to evaluate the formulae 
and how to check the results. 

A formula for a calculation to be carried out numerically has a 
significance rather different from that of a formula in formal algebra or 
analysis. For example, the formula 

y = {x^-\-l)l2x (3.1) 

regarded as an algebraical formula states a relation between the quanti¬ 
ties on the two sides of the sign of equality, and is completely equivalent 

x^—2yx-\-\ = 0 

or x = y±{y^—\f, (3.2) 

which are different ways of expressing the same relation. But formula 
(3.1) regarded as a formula for a numerical calculation specifies a process 
to be carried out for determining the value of y given the value of x, 
whereas formula (3.2) specifies a process to be carried out for deter¬ 
mining the value of x given the value of y. These processes are different 
from one another, the data used in them are different, and the results 
required are different. This aspect of a formula, as representing a process 
consisting of a set of operations to be carried out in a definite sequence, 
plays little part in formal analysis, but is fundamental in numerical work. 
Even the formulae 

X = y—z and z = y—x 

mean quite different things when regarded as specifications of numerical 
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calculations to be carried out; and the process specified by the formula 

y — i[^+(l/^)] (3.3) 

is different from that specified by formula (3.1). 

A striking example is discussed in § 11.3, where it is shown that of 
two ways of writing the recurrence relation for the Bessel functions, 
namely, J„+i(a;) = {‘^njx)J^{x)—J^_^{x) 

Jnix) = {xM{J^^^{x)-\-J^_^{x)l 

which are formally completely equivalent (for w > 0, a; > 0), the first 
specifies a numerical process which is quite impracticable as a general 
method for calculating JJ^x) for w > a: > 0, whereas the second gives a 
quite practicable iterative process. 

There may be various ways of evaluating even simple formulae, and 
the best way may depend on the equipment available for carrying out 
the numerical work. For example, in the evaluation of {abc...)l{uvw...) 
by means of a slide rule it is best to take multiplications and divisions 
alternately, expressed by writing this fraction in the form 

[{{alu)Xb}/v]xc .... 

But with a desk machine it is best first to evaluate the denominator 
D = uvw... and record this, then form the continued product abc..., and 
finally divide the result by D. In forming these continued products, no 
intermediate results need be written down; the only numbers to be 
recorded are D and the final result. 

In using a machine it is worth while planning the calculation in such a 
way that as much as possible of the work is done on the machine without 
recording intermediate results, so as to reduce the amount of writing, with 
the possibilities of mistakes in recording and reading the written results, 
to a minimum demanded by the need for clarity in presentation of the 
calculation and for checking. Transfers from the counting register to 
the setting levers or keys should also be avoided if possible. 

For example, if e* were given, 2(cosha;—1) could be calculated from 

2(cosha;—1) = e^-{-{l/e^)—2; 

this would require a reciprocal to be calculated, recorded, and reset on 
the machine (or at least transferred from the counting register to the 
setting levers or keys). But if it is calculated from 

2(coshx—1) = (e^—l)“/c*, 

this can all be done by a sequence of operations on the machine alone; 
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it also has the advantage that for small x it does not calculate the result 
as the small difference of two relatively large quantities. 


3.2. Evaluation of polynomials 

Expressions consisting of a number of additions and multiplications 
can usually be evaluated in various ways, of which the best to use in 
any case may depend on particular features of that case. For example, 

a polynomial y = a„x’>+a,x’'-^+...+a„.^x+a„ (3.4) 

may be evaluated by calculating the separate terms and adding. When 
X has a simple numerical value {x = 1, 2, or 10 for example) this may be 
the best method, especially if the coefficients are small integers. If this 
method is used for evaluating a polynomial both for positive and for 
negative values of x, a convenient procedure is first to sum separately 
all the terms involving odd powers of x and all those involving even 
powers of x, for positive values of x only, then for each value of x to add 
and subtract these two sums. 

If X has not a simple numerical value it may be better to write 


y = [{K^+%)^+«2}^+“3]^+— (3-5) 

and carry out an addition and a multiplication alternately as indicated 
by this expression. That is, construct the sequence defined by 


yo = «o. y,-= (i>0); (3.6) 

the result required is y^. This process requires n multiplications and n 
additions, and no recording of intermediate results. Care is necessary 
with the decimal point; use of the decimal point markers is a great help 
here. 

The process for checking the results will depend on the calculation of 
which the evaluation of the polynomial forms part. It is unlikely that 
just a single value of a polynomial will be wanted; the evaluation of the 
polynomial is much more likely to form part of a larger calculation, which 
may well include means of checking the value obtained for the poly¬ 
nomial. 

If a set of values y of a polynomial (3.4) for a set of values of x is cal¬ 
culated, then 




(3.7) 


where the sum is over all values of x for which the poljmomial has been 
calculated. One way of checking such a set of values of y is to evaluate 
the right-hand side of (3.7) and compare the result with y\ the results 
should not differ by more than the tolerance for rounding errors. 
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We shall see later (§ 4.42) that if a peljoiomial has simple coefficients 
and is of not too high order, its values for a set of equally-spaced values 
of X can be obtained simply and conveniently by a sequence of additions, 
without any multiphcation at all. 


3.3. Evaluation of power series 

To evaluate the sum of a power series 

y = ao+aiX-fagX^-f... (3.8) 

it is often most convenient to write each term as a multiple of the 
preceding one, thus: 

and to evaluate each term from the previous one by the appropriate 
multiplications. Series containing odd powers only or even powers only 
can be treated similarly. If several values of y, at equal intervals of x, 
are calculated, evaluation of the finite differences (§ 4.2) of the values 
of y probably provides the best check. 


Example: To evaluate 


y = 


1 


1 


2.4.5 ' 2.4.6.7.8 

to six decimals for x = 10(0-1)1'4. 

It is convenient to write this 




1 


2.4.6.7.8.10.11 




S' = ^•+ 4.6.7.8.10.11 ^'+-] 


(3.9) 


and first to sum the series in the square bracket and then multiply the sum by 
If the ratios of successive coefficients in the scries are written fe-j,... then 


(nth term) = 6yjX^[(n—l)th term]; 
the values of the first few 6’s are 


(3.10) 


h A _ J_^l 

^ 4.5 20' ^ 7.8 56' 


1 


1 


10.11 110 '***' 


and in general 


b ^ ^ 

^ (3n+l)(3n + 2)* 


The denominator in this fraction is a quadratic function of n, hence the second 
differences (see § 4.2) of its values are constant, and this can be used to check these 

values 20 66 110 182 272 380 

36 64 72 90 108 

18 18 18 18 


A similar check can usually be applied if the ratios of successive coefficients can be 
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expressed as the ratio of two polynomials of low degree in n. The work can 
conveniently be arranged as follows: 


X . . . 

1-0 

1-L 

1-2 

1-3 

1-4 

. 

1-000 

1-331 

1-728 

2-197 

2-744 

K 

1-000000,00 

1-000000,00 

1-000000,00 

1-000000,00 

1-000000,00 

1/20 = 05 

0050000,00 

0-066550,00 

0-086400,00 

0-109850,00 

0-137200,00 

1/56 = 01785714 

892,86 

1581,75 

2666,06 

4309,65 

6722,80 

1/110 = -00909091 

8,12 

19,14 

41,88 

86,08 

167,70 

1/182 = -005495 

0,04 

0,14 

0,40 

1,04 

2,53 

1/272 = -003676 

0,00 

0,00 

0,00 

0,01 

0,03 

sum . 

1-050901,02 

1-068151,03 

1-089108,34 

1-114246,78 

1-144093,06 

. 

0-5 

0-605 

0-72 

0-845 

0-98 

y (to six decimals) 

0-525451 

0-646231 

0-784158 

0-941539 

1-121211 



17147 

19454 

22291 


y" = 1 -fxy 

1-52545 

1-71085 

1-94099 

2-22400 

2-56970 

. 


4474 

5287 

6269 


h^y" . 



169 



2/’+iW-2io8V 


1-71458 

1-94539 

2-22922 



Notes: (i) The entries in the third to eighth lines are the values of the terms in the square 
bracket in formula (3.9). Each is calculated from the preceding one by formula (3.10 ); 
if the decimal values of given on the left are used, these terms can be calculated 
entirely by multiplication and transfer. 

'(ii) To obtain six decimals in the final result, it is advisable to keep eight decimals in 
the individual terms, that is, to retain two guarding figures. 

(iii) The function y defined by the series (3.9) satisfies the equation y" ~ \-\‘Xy, The 
second differences (see § 4.2) of y can bo calculated from tho values of y (see § 4.45) and 
compared with the values calculated from y" by formula (4.19); this provides a close 
check on the results. 

3.4. Kinds of formulae to avoid 

There are two kinds of formulae to be avoided if possible, namely 
those that express the result required as 

(i) the ratio of two small numbers, 

(ii) the difference of two large, nearly equal, numbers. 

When one or other of these situations occurs, it often, though not always, 
means that the method adopted for calculating the result is not the most 
suitable, and it is usually worth examining whether there is a more 
suitable alternative. 

The following are some examples: 

(a) Exponential extrapolation 

Three numbers y^, y^, and y^ are known to differ from the required 
result Y by amounts which are in geometrical progression; to find Y (see 
Fig. 5). This process is ealled ‘exponential extrapolation’; it is useful in 
some methods of successive approximation (see § 9.32). 
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Since yf^—Y, y^—Y, and 1/2— Y are in geometrical progression, 
(yz-mVi-Y) = {y,~Y)l(y,-Y) 
and solution for Y gives 

Y = (2/o2/2-!/i)/(2/2-2!/i+2/o). (3.11) 

But if y^ = y^ = y^ this gives Y = OjO which is useless for numerical 
work; and if y^, y^, and y^ are only slightly different from Y, it gives Y 
as the ratio of two small numbers, the numerator and denominator 
being both of order (y^—T). 

But if Y is written as the best approximation y^ plus a correction, 
5^ = 2 / 2 —( 2 / 2 —yi)V(y 2 —2yi+yo). (3-12) 

the numerator of the ‘correction’ is of order (^q— Y)^ whereas the 
denominator is of order {y^ —F); the correction is therefore of order 
(yo— Y) and is zero in the case yQ = y^ = y^, and is small if yo is nearly 
equal to Y. Its evaluation gives no trouble. 

This illustrates the way in which two expressions, formally equivalent, 
may be very different when assessed from the point of view of the ease 
of practical numerical evaluation. 


I 


y 

o 



Fig. 5 


( 6 ) Solution of a quadratic equation when the ratio of the roots is large 
Let Xi be the larger and Xg the smaller of the roots of the equation 

a:2—18a:+l = 0. 

Use of the standard general formula for the root gives 

Xi, X 2 = 9iV80. (3.13) 

If V80 is taken to four decimals (five figures) this gives 

X 2 = 17’9443, 0*0557. 

Here X2 is obtained as the small difference of two relatively large numbers 
9 and V80 = 8*9443; the first two significant figures in the value of V80 
are lost, and from a five-figure value only a three-figure result is obtained. 
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On the other hand if Xg is obtained not from (3.13) but from the 
relation (for this equation) = 1, the value of is obtained to full 
five-figure acciu’acy without requiring that V80 should be obtained to 
any greater accuracy than for x^. Here again we see a marked difference, 
from the point of view of numerical evaluation, between two formally 
equivalent formulae. 

3.5. Evaluation of a function in the neighbourhood of a value of 
the argument at which it becomes indeterminate 

In the neighbourhood of a value of the argument at which a function 
becomes indeterminate, some form of series expansion will usually be 
available. 

Consider, for example, the function y defined by 


y = (1/sina:) —(l/x) (0 < |a:| < n) 

2 /( 0 ) = 0 . 


To evaluate this for small values of x, it is convenient to write it 




X 


6 


It would be possible to carry out the division of one series by the other 
algebraically, but if more than the first two or three terms have to be 
included, it is easier to evaluate the two series separately and carry out 
the division numerically. 
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4.1. Functions of a continuous variable in numerical analysis 

In numerical work we may be concerned with two different ways of 
specifying functions of a continuous variable. First, a function may be 
specified by a formula which can, in principle, be evaluated for any value 
of X as required: examples of such functions are polynomials, circular, 
exponential, and other functions defined or expressed in terms of conver¬ 
gent power series, and functions defined by definite integrals such as 
the gamma-function „ 

r(a;) = J dt. (4.1) 

0 

Secondly, there are those functions which are specified only by tables of 
values; these may often be tables expressing some empirical physical 
relationship, such as the relation between grid voltage and anode current 
in an electronic valve, or between velocity and resistance for a projectile; 
or they may be results of previous calculations. 

In practice, there is not much difference between functions specified 
in these two ways, for usually one obtains values of functions of the first 
kind from tables rather than by evaluating the defining formulae. In 
fact, mathematical tables are made precisely for the purpose of enabhng 
function values to be determined without going back to first principles 
and evaluating the defining formulae each time a function value is 
required; if we require r(l'2783C) we interpolate in tables of r(x), 
rather than evaluate the integral in formula (4.1) for x = 1-27836, 
unless it happens that no tables to the number of figures required are 
available. Thus in either case we are concerned in practice with functions 
specified by tables, and with the properties of functions so specified. 

A function J{x) specified only at discrete tabular values of the inde¬ 
pendent variable x is not formally defined for intermediate values. If 
the tabular values of x include zero and are at equal intervals 8x, and 
g{x) is any function (not necessarily even continuous) which is finite 
at the tabular values of x, then f{x)+g{x)&m.{TTxlhx) has the same values 
as/(a:) at the tabular values of x. Further, the tabular values of f{x) are 
usually subject to rounding errors, so that the function may not be 
accurately defined even at the tabular values of x. 

On the other hand, a table of a function of a continuous variable x 

D 


6013 
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would often be of little value unless it were possible to determine values 
of the function for values of a: between the tabular values (to an approxi¬ 
mation depending, of course, on rounding errors). In order to do this, 
some understanding is necessary about the behaviour of the function 
between its tabular values, an understanding which may be justified 
formally in cases of functions of the first kind mentioned at the beginning 
of this section, but may have to remain an assumption in the case of 
empirical functions. This understanding may be expressed quahtatively 
by saying that the function is ‘smooth’ over the range concerned. 
‘Smoothness ’ of a function is a property which it is difficult to define in a 
quantitative way; it is discussed further in § 11.4. It implies differentia¬ 
bility to some high order, and smallness of high-order derivatives. An 
example will illustrate this. 

We shall later (Chapter V) derive interpolation formulae for use when the interval 
of tabulation is too large for linear interpolation between tabular values of the 
function. It will be found that it is possible to interpolate sin x, not only roughly 
but to any required accuracy, from its values at interval Sx = 

X —§77 —77 —^ 0 +^77 77 §77 j 

y = sinaj 1 0 —1 0 1 0 —l/ l-l 

or even from its values at intervals x — § 7 t. 

Let us inquire what particular property the function y = sin x has which selects 
it from all other functions with the tabular values (4.2) as the one for which the 
interpolation formula is accurate. Suppose, for simplicity, that we know that y is 
an odd function of x, periodic with period £ 77 . Then it can be expanded in a sine 
series in the interval —77 < a; < 77 : 

y — sin a;-f sin 2a;-faj sin 3a;-f... (4.3) 

where, to give the value of 7 / at a; = 

1 = a^—aj-faj—a,-f.... (4.4) 

W e shall require a measure of the nth derivative of 7 /; this derivative varies with x, 
but a convenient single quantity giving an overall measure of its magnitude is its 
mean square value 

7T 

~ J (y<">)^dx = ^(aH2’'ai+3”aH-...). 

— TT 

As n 00 , the minimum value of this quantity, subject to the condition (4.4), is 
given by = 1, = 0 (m > 1). Thus the relevant property of the function 

2/ = sin a; is this, that of all functions which are odd and have period 277 , it is the 
one for which, to put it roughly, the high-order derivatives are as small as possible. 

Another aspect of this property of the function sin a:, that it can be interpolated 
accurately from its values at a wide interval such as ^77 or ^^ 77 , is considered in § 5.91. 

In the great majority of cases, functions are tabulated at equal 
intervals of the independent variable, which is often called, in this 
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context, the ‘argument’ of the table. For the present, we will only 
consider such sets of function values. 

4.2. Finite differences 

The most important property of a function specified by a table con¬ 
sists of what are called its ‘finite differences’. The following example 
illustrates what is meant by this term: 




First 

Second 

Third 

X 

f(x) = l/x 

differences 

differences 

differences 

3 0 

0 33333 

-1075 



3-1 

•32258 

-1008 

67 

-6 

3*2 

•31250 

— 947 

61 

-5 

3-3 

•30303 

- 891 

56 

-6 

3*4 

•29412 

— 841 

50 

-2 

3-5 

•28571 

— 793 

48 


3’6 

0-27778 





The ‘first differences’ are obtained by subtracting each function value 
from that for the next greater tabular value of x‘, the ‘second differences’ 
arc obtained by carrying out a similar set of subtractions on the first 
differences, and so on. 

Values oiodd order differences should be written on levels intermediate 
between those of function values, and values of even order differences on 
the same lines as function values; normally these values are written in 
terms of the last digital position as unit, decimal points, and zeros before 
the first significant digit, being omitted. They can conveniently be dis¬ 
tinguished from function values by being written or printed smaller. 

The finite differences of tabular functions play a very important part 
both in the analytical and in the numerical manipulation of such func¬ 
tions. Use of them enables formulae for operations on such functions, 
such as interpolation and integration, to be expressed compactly and in 
a form convenient for practical use. When the tabular values provide all 
the information we have about a function, all processes involving this 
function have to be expressed as operations on the tabular values; one 
of the most important operations on a set of values at equal intervals of 
the independent variable is that of differencing, and we shall see, later 
in this Chapter and in Chapters V and VI, that most of the other 
operations can be expressed in terms of this one. 
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It wiU be seen that the values of the third differences in the above table 
are noticeably irregular; this is an effect of rounding errors in the function 
value, which will be considered more fully in § 4.44. 

In order that a function shall be well determined by a table, the 
average value of the wth order differences should tend to zero, or at 
least become small, as n increases. We have seen examples in which this 
does not appear to be the case; for the function y = sin a:, at interval 
^TT in X (see (4.2)), the 2wth differences have extreme values ±2”^, but 
this function is still well defined by these values, in the sense that accurate 
intermediate values can be interpolated between them. But this is a 
peculiar property of the function y — sin x alone out of all the functions 
with these tabular values; given these function values alone, without the 
knowledge that they are intended to represent sin x, one could not be at 
all confident about the results of any attempt to interpolate between 
them. Eight values per cycle is about the smallest number which can in 
practice be regarded as specifying an oscillating function adequately, 
and at least twelve values per cycle is preferable. 

4.21. Notation for finite differences 

Let X(f be one of the tabular values of x, Xj — (xQ+i Sx) a set of other 
tabular values, and/,- = f{Xj) the values oif{x) at the tabular values of a:. 

There are two kinds of notation for finite differences. In one the 
differences of a function / are written 8/ or A/, so that the symbol 8 or A 
stands for an operation carried out on the values of the function/. In the 
other the symbol 8 or A is used for the differences themselves. 

The former seems much the preferable, both for use in the derivation 
and manipulation of formulae in finite differences and in application of 
them. It is more nearly self-explanatory, and many formulae with which 
we shall be concerned express relations between differences of two different 
functions (for example a function of x and its derivative), and if a symbol 
is used to represent a difference itself rather than a difference-operator, 
differences of different functions cannot be distinguished except by 
introducing new symbols, which are unnecessary. In this notation, 
repetition of an operation is expressed by the use of an index (as in 8^, A®), 

The use of the symbol 8 or A for the differences themselves is a con¬ 
venient shorthand in cases in which it is unambiguous, and is sometimes 
preferred by those carrying out the details of the numerical work. In 
this notation the use of dashes (A") or Roman superiors (as A^^"^^) is prefer¬ 
able to the use of numerical indices to indicate orders of differences. 

In this book, the former usage wiU be adopted throughout, so that 8 
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and A must be regarded as finite-difference operators. Consistently with 
this notation, hx will be used for the interval in the independent variable 
(other notations for this are h and w).'\ 


The first difference/ i—/q may be associated with the argument value 
Xq, with the argument value x^, or symmetrically with these two argu¬ 
ment values, and assigned a corresponding suffix. A different symbol for 
the finite-difference operator is used to distinguish these three cases: 

fi fo ~ A/o' 

= y/j. (4.5) 

= S/*i 

This is generalized in the following three schemes for a difference table: 


* / 

A*/-. 

A/-. A*/-, 

A*/-a 



/ 



Backward differences 



6’/-, 





fi/-i 



«•/. 

«/i 



■ 








(r) 




Central differences 


In any particular numerical case the numbers wfil be the same in each 
table; what is different is the general notation for these numbers, the 
notation which expresses the value of x with which each difference is 
associated. 

Differences with the same suffix value in table (a) are called ‘forward 
differences’; they lie on a downward-slanting line on the table, such as 
those underlined. The forward differences from the first entry in a table 
are sometimes called ‘leading differences’. Those differences with the 
same suffix value in table (6) are called ‘backward differences’; an 
example is indicated similarly. Those with the same suffix in table (c) are 
called ‘central differences’. 

Central differences are much the most useful in practice. Many 
formulae in central differences involve only alternate orders of differ¬ 
ences, whereas the corresponding formulae in forward or backward 
differences involve all orders of differences; also the coefficients of 
higher terms in central-difference formulae usually decrease more rapidly 
with the order n of the differences than do the coefficients in formulae 
involving forward or backward differences. Further, this notation gives 
a much more natural relation between finite differences and derivatives. 


f It is sometimes convenient to distinguish between the general symbol So; for the 
interval length and the particular value which it has in a particular calculation. 
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In the analytical work of deriving formulae for interpolation, integra¬ 
tion, etc., in terms of differences, use of forward differences leads to 
rather simpler algebra; but in order to get from the results the central- 
difference formulae which are most convenient for practical use, it may 
be necessary to do some rather laborious algebra, which may then only 
give the coefficients of the central-difference formulae term by term, and 
be difficult to generalize to give the general term. It seems best to work 
throughout in terms of central differences, and so obtain directly the 
formulae for interpolation, integration, etc., in the forms in which they 
are most useful for practical work. The symbol A is then left free for 
another use, to indicate the difference between the data or between the 
results of two similar calculations. 

It will be seen that in the central-difference scheme (c) on p. 37, only the 
even-order differences have integral suffixes. It is sometimes convenient 
to take the arithmetic mean of two adjacent differences and to write 


and in general 






Then the available differences are odd-order differences with (integer-j-^) 
suffixes, and even-order differences and odd-order mean differences with 
integral suffixes. A set of successive function values from j — J—k 
to J-\-k inclusive is said to be ‘centred on’ the argument value Xj or on 
the function value fj\ similarly for a set of differences 8”/;. 


4.3. Finite differences in terms of function values 

It is sometimes convenient to have differences expressed in terms of 
the function values from which they are derived. We have in succession 

8/i =/i (4-C) 

8^0 = 8/j-8/_j = 

= — (4.7) 

8^* = 82/,-82/o = (/2-2A+/o)-(/i-2/o+/-,) 

=/2 ~ -f 3/0 —/_J, 

and in general S«/, = ^ (4.8) 

as can be proved by induction; for an alternative proof see § 4.6. The 

coefficients of the function values in 8"/y arc those in the binomial 
expansion of (1— 2 )". 
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In particular, the differences of the function 

/m = 0 W ^ 0 

fo — ^ 

are the binomial coefficients: 


X 

/ 

8 / 

8 */ 

8 ’/ 

sy 


0 

0 

0 

1 

1 


0 

1 

1 

-3 

-4 


1 

— 1 

-2 

3 

6 

^1 

0 

0 

1 

— 1 

-4 


0 


0 


I 


The effect of an error e in a function value on the difference table builds 
up in the same way: 

Oi 

X_i 

Xt 

This is the basis of an important application of differences to checking 
tables, and sometimes for correcting isolated errors, which will be con¬ 
sidered shortly (§ 4.43). 

4.4. Simple applications of differences 

The simplest applications of differences are: 

(a) Building up polynomials; 

(b) Checking tables; 

(c) Smoothing. 

Of these, (a) and (b) will be considered here and (c) in § 11.4. 

4.41. Differences of a polynomial 

An important property of finite differences is that for a polynomial of 
degree n, the nth. order differences are constant. One proof of this is by 
induction. 

Suppose that 8’”(a;”‘) = m! (Sx)”* 



(4.9) 
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for all integral values of m up to m = n, say; then it will be proved that 

(4.9) holds for m = n+2. Since for p < m 

8«(a:P) = 8’"-P(SPzP), 

(4.9) implies that 8 ”*(.t?') = 0 forp<m. (4:.10) 

Now from (4.7) 

32 (^n+ 2 j _ (a;-j-§a:)”+2— 2x'^+^-{-{x —8a:)”+2 

= (n+2)(n+l)(8a:)V‘+terms of lower degree, 
so 

Sn+ 2 ^n +2 _ 311 ( 32 ^ 71 + 2 ) 

= (?i+2)(?i+l)(8.r)2S”[a:”+terms of lower degree] 

= (?i+2)(?i+l)(8a:)^S’^(a:”) [by (4.10)] 

= {n-\-2){n-\-\){8xYn\ {8xY [since (4.9) holds for m = ri] 

= (71+2)! (8:c)”+2. 

Now (4.9) holds for tti = 1 and 7 /i = 2, hence the induction succeeds, and 

(4.9) holds for all integral 7ti. 

It follows that for a polynomial of degree m, say 

Vrni^) = 

the 772th differences are constant and equal to a^TTz ! (8a:)"'. 

Another derivation, which depends on some properties of a set of functions to 
which reference will be made later, is as follows. Consider the polynomials:! 

UO = b 

m -1 

= n [^+{i(w-l)-A:}]; (4.11) 

fc = 0 

) is a polynomial of degree m in and its argument value is the mean of the 
extreme factors. The first few such functions are 

uo = b 

<Ai(f) = t 

UO = (^+i)(^-i) = 

= 

For intervals 8^ = 1, the first differences of are 

■= [^+ Urn+ l)][^+i(m-1 )]...[f- ^(m- 3)]- 

- [f+i(m-1)]...[^- l{m~ 3)][f- i(m-1)]. 

>— _ - ^ 


t Sometimes called ‘factorial polynomials’. 
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The conunon factor of the two terms, indicated by a bracket, is a polynomial of 
the set (4.11); it has (m—1) factors and the mean of its extreme factors is (f+i). 
so it is +i). Hence, for m > 0, 

and 8^. -0. " 

Repeating the operation we have 

~ —i) 

= 0. 

Any polynomial p„,(a;) = <3o^”*+®i^”*~* + ”-»tabulated at intervals (8x) in Xy can 

Pm(^) = ®o(S^)”*[!/'m(^) + 6l^m-l(^) + &2^m-2(^) + —] 
where ^ = x/8x, so 8”*Pm(^) = aginl (8x)”* 

as already shown. 

This result, that the mth differences of any polynomial of degree m are constant, 
and its (m +1 )th differences are zero, corresponds, in finite differences, to the result 
in differential calculus that the mth derivative of such a polynomial is constant and 
its (m +1 )th derivative zero. The functions take the place, in finite differences, 

of the functions x^ in differential calculus, as the polynomials whose form remains 
imchanged on differencing. 

These functions will appear later in another context, for which some further 
properties of them will be required. From the definition (4.11) it follows that 

M-i) = 

Hence for odd values of m 

*p2n+l(i)'h^2n+l(^ i) ~ *p2n+l(.i) ^2n+lii 1 ) 

= {2n-\-l)tp2n{i~h) (4.13) 

by (4.12), whereas 

^2n+l(^) *p2n+l(^ ~ ^2n+l{^)'^^2n+l{^ 1 ) 

= [(f+n) + (f-n-l)].^,„(f-i) 

= {^i-l)4>2n{i-i)- (4.14) 


4.42. Building up polynomials 

The constancy of the mth differences of an mth order polynomial can 
be used to construct a table of values of the polynomial by building up 
successively the lower orders of differences from the higher by repeated 
addition. It is necessary to calculate at least m function values to give 
a set of leading differences from which to start the construction of the 
difference table, and it is advisable to take one or two more to provide 
a check. 
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Example: 

To evaluate the polynomial y — 

1 for a; = 

= 0(1)10: 

X 

X* 

— 5x2 

-l-6x 

4-1 = 

y 


82y 


-2 

-8 

-20 

-12 

4-1 = 

-3^ 

4-28^ 



-1 

-1 

-5 

-6 

4-1 = 

-11 

12 

--7^ 

6 ) 

0 

0 

0 

0 

+ 1 = 

1 


-10 

1 start 







2 


6 ) 

1 

1 

-5 

6 

4-1 = 

3 

-2^ 

-4 

6 

2 

8 

-20 

-f 12 

4" 1 = 

1 

0 

2 

6 

3 





1 

8 

8 

6 

4 





9 

22 

14 

6 

5 





31 

42 

20 

6 

6 





73 

68 

26 

6 

7 





141 

100 

32 

6 

8 





241 

138 

38 

6 

9 





379 

182 

44 


10 

1000 

-500 

-f 60 

4-1 = 

561 



check 


Here five function values, from x = — 2 to 2 (the simplest ones to evaluate) have 
been calculated to provide a start for building up the differences. We know from 
(4.9) that the third differences must have the constant value 6, and this provides 
a check on the starting values. From the constant third differences of 6 the second 
differences are built up, then the first differences, and finally the function values. 
The function value at a; = 10 is easy to calculate directly, and is so calculated to 
provide a check on the successive additions. 

It will be noted that intermediate values of y are calculated by addition 
only: this process can be carried out very effectively on an adding 
machine fitted with a printing mechanism (§ 2.26). For example, in 
summing the second differences to give the first differences, after adding 
each second difference the resulting value of the first difference, which 
is the current total, is printed without clearing by taking a ‘sub-total’. 
The results appear in the form of alternate values of second differences 
and first differences; the former can be checked against the table of 
values and the latter then summed similarly to give the function values. 

It is necessary in using this process to keep all figures without rounding 
off, although final results may not be wanted to this accuracy. 
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Example: To evaluate the polynomial y — x^ 

-5xH6x+1 

for X 

= 0(0*01)0-1; four 

decimals required. 














y rounded off to 


X 

+ 6x +1 

= y 



'four decimals 



-0-02 --0000,08 --0020 

-012 +1 

= -8779,92 

61507 



-8780 

615 


0-01 --0000,01 --0005 

-0-06 +1 

= -9394,99 

60501 

-1006 

6 

-9395 

605 

- 10 

0 0 0 

0 +1 

= 1-0000,00 

59501 

-1000 

6 

1-0000 

595 

- 10 

0-01 +-0000,01 --0005 

+ 0 06 +1 

= 1-0595,01 

58507 

- 994 

6 

1-0595 

585 

- 10 

0-02 +-0000,08 --0020 

+ 0-12 +1 

= 1-1180,08 

57519 

- 988 

6 

1-1180 

575 

- 10 

0-03 


1-1755,27 

56537 

- 982 

6 

1-1755 

566 

- 9 

004 


1-2320,64 

55561 

— 976 

6 

1 2321 

"^65 


005 


1-2376,25 

54591 

- 970 

6 

1-2786 

.^6 

+ 171 

006 


1-3422,16 

53627 

- 964 

6 

1-3422 

536 

^100 

007 


1-3958,43 

52669 

- 958 

6 

1-3958 

527 

- 9 

003 


1-4485,12 

51717 

- 952 

6 

1-4485 

517 

- 10 

009 


1 5002,29 

50771 

- 946 


1-5002 

508 

^ 9 

0-10 


1-5510.00 




1-5510 




Notes: (i) Although the third difference of 6 in the sixth decimal is smaller than 
the rounding error in the four-decimal values finally required, it must not be 
neglected on that account, as this would be a systematic rounding error which would 
accumulate and ultimately affect the results wanted. Omission of it would be 
equivalent to omitting the term in the polynomial, and the error would already 
be 10 in the fourth decimal at x = 0*1. 

(ii) Here a typical copying mistake (78 for 87) has been made in the column of 
rounded-off values, which are those finally required. Such a mistake is easy to 
make at this stage; all the calculations have been done, and all that is wanted 
is to copy the four decimals required with the appropriate rounding off; uncon¬ 
sciously one may relax some of the care with which the rest of the calculation has 
been carried out, and then a mistake of this kind can easily occur. Such a mistake 
is easily identified by differencing the rounded^off results and such a check should 
always be used. As will be seen in the following section, the irregular differences 
not only locate the erroneous value unambiguously, but strongly suggest the 
correction. 

4.43. Checking by differences 

We have seen in § 4.3 that an isolated error e in a function value makes: 
a maximum error e in the first differences 

2e second differences 

3e third differences 

6e fourth differences 

lOe fifth differences 

20e sixth differences 
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whereas the magnitude of the differences themselves normally decreases 
with the order of differences; if it does not, the function is not well defined 
by the table. Hence an error shows up more and more as the order of the 
differences is increased. Examination of the differences of a function is 
one of the best checks against random errors; it will not necessarily check 
against systematic errors. 

The differences which are affected by an error spread fanwise from the 
incorrect function value (see § 4.3), and this can be used to locate an error. 


Example: 













corrections 

X 

y 


hhj 


sv 

to B*y 

0 

358 








12 





1 

370 


15 






27 


12 



2 

397 


27 


— 1 




54 


11 



3 

451 


38 


- 1 




92 


10 



4 

643 


48 


- 1 




140 


9 



5 

683 


57 


- 

+ 18 



197 


- 



6 

880 


"47 


+ 71 

- 72 



244 


+ 61 



7 

1124 


108 


-109 

+ 108 



352 


-48 



8 

1476 


_60 


+ 71 

- 72 



412 





9 

1888 


83 


- 19 

00 

+ 



495 


4 



10 

2383 


87 


- 1 




582 


3 



11 

2966 


90 






672 





12 

3637 







The last column is 18 X (1, —4, 6, —4,1). 


Notes: (i) The existence and location of an error is unambiguously shown by the 
table. 

(ii) A change in a function value y makes changes (1, —4, 6, —4,1) times 

in successive values of the fourth difference, centred on the changed value of y. 
A few trials show that a change A^/ = +18 will make all the fourth differences — 1. 
The error can often be corrected in this way. 

(iii) A transposition of two adjacent digits differing by m will produce an error 
of 9m in terms of the less significant of the digits as unit. It has already been 
mentioned that transpositions form a common type of mistake; values of A^/ which 
are multiples of 9, or nearly, probably arise from mistakes of this kind. This can 
be checked from the values of the digits involved. Here Ay — 18, hence m = 2 
in the last figure. The value y = 1124 at x = 7 should read y = 1142. 


FINITE DIFFERENCES 


45 


(iv) In this case the fourth difference of the corrected table is exact; the location 
and correction of the mistake is not affected by rounding errors. 

Example: Here the values of y are alleged to be rounded off from a table of x^: 




hy 


Correct ion Revised 

Correction Corrected 

X 

y 

8 *2/ 

to 8^y 8^y 

to 8^y 8^y 

38 

6-1644 

806 






39 

•2450 

796 

-10 


-10 


-10 

40 

•3246 

785 

-11 


-11 


-11 

41 

•4031 

776 

-9 


-9 


-9 

42 

•4807 

^740 

-36 

+ 27n 

-9 


-9 

43 

•5547 


+ 45 

-54 

-9 


-9 

44 

•6332 

750 


+ 27> 

-8 


-8 

45 

■7082 


"'"O 

-O'! 

-9 


-9 

46 

•7832 

745 

-5 

+ 18 

+ 13 

-20\ 

— 7 

47 

•8577 


— 40 

-9; 

-49 

+ 40 

-9 

48 

6-9282 

718 

.<M3 


+ 13 

-20; 

— 7 

49 

7-0000 

711 

-7 


-7 


-7 

50 

•0711 

703 

-8 


-8 


-8 

51 

•1414 


-6 


-6 


-6 


52 -2111 

Notes: (i) Here a succession of seven values of is irregular. The first obviously 
wrong value, — 36 at x = 42, indicates a mistake at x = 43; the value of S^2/(42) 
would be expected to be —8, —9, or —10; that is, the correction is +28, +27, 
or +26. The value +27 suggests a transposition of two digits differing by 3 in 
2/(43), and reference to the function values shows that the end digits do differ by 3. 

(ii) Correction of this mistake then makes the differences smooth, apart from 
slight irregularities which can be ascribed to rounding errors in the fimction values, 
as far as 8^2/('^4) inclusive. The next four second differences are irregular, indicating 
mistakes in both the values 2/(46) and 2/(47). The value of8^y{46) would be expected 
to be — 8 or — 9; that is the correction is — 8 or — 9; the latter suggests an inter¬ 
change of two digits differing by 1, and when the corresponding correction has been 
made, we have the series of second differences given in the column headed ‘ Revised ’. 

(iii) The next three second differences should be about — 8, and to give them all 
this value we would require corrections ( — 21, +41, —21); the corrections arising 
from a single change in y must be in the ratio (1: — 2:1), so the error in 2/(47) is 
+ 20 or +21. The former would be produced by doubling the wrong one of two 
digits differing by 2, and as such digits do occur in 2/(47) in the right place, the 
error can be ascribed to this cause with fair certainty. 
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These examples show that it is possible to use differences not only for 
detection and location of errors in tables, but for correcting them, when 
the nature of the error is clear from the behaviour of the differences, or 
for indicating a probable correction when it is not. In the case of the 
second example just given, it would of course be much better to use the 
differences simply to indicate the erroneous values, and to refer back to 
a table of to correct them. 

4.44. Effect of rounding errors on differences 

In most tables almost every function value will be in error to some 
degree, on account of rounding errors. Although the rounding error in 
a function value may not be more than | in the least significant figure, 
the effect of an error is exaggerated in the higher differences, which un¬ 
avoidably become somewhat irregular, and the more so the higher the 
order of differences. It is important to realize this, otherwise irregularities 
in differences which are due to rounding errors may be taken as indicating 
mistakes, and time may be spent trying to find mistakes and to make 
changes in function values which cannot be improved except by taking 
more significant figures. 

The greatest effects Of rounding errors will occur when alternate func¬ 
tion values are rounded off by -t-|^ and — | alternately. Then departures 
of the wth differences from those for unrounded function values may be 
up to 2”“^ in the last place tabulated and alternate departures will be 
of alternate signs; though such large irregularities will be rarer the higher 
the order n of the differences. It is useful to have a working criterion 
for the magnitude of the fluctuations in the different orders of differences 
which can be expected as the result of rounding errors. Comrief gives 
the following limits for various values of n: 

n 1234568 10 

±1 ±2 ±3 ±6 ±12 ±22 ±80 ±300 

Differences having fluctuations less than these limits can be accepted; 
only those having greater fluctuations should be regarded as suspicious. 

The example opposite illustrates the w ay in which irregular differences 
may occur in the most accurate rounded-off values of a smooth function. 

From examination of the differences in the table, one would be very 
inclined to ‘correct’ the rounded values of y(6) and y(8) to 393 and 777 
respectively, giving the third differences shown on the extreme right; 
but reference to the exact values of y shows that this would be incorrect. 

t Chambers's 6-Figure Mathematical Tables, vol. 2 (1949), Introduction, p. xxxi. 
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Values of y rounded ‘ Corvee* 
off to nearest unit tions'" 


X 

y 

8y 

B^y 

B^y 


y 

Sy 

B^y B^y 

to S^y 

h^y 

0 

61-24 

4581 




61 

46 





1 

107-05 

5478 

897 

109 


107 

55 

9 

1 


1 

2 

161-83 

6484 

1006 

162 

53 

162 

65 

10 

1 


1 

3 

226-67 

7652 

1168 

214 

52 

227 

76 

11 

4 

-1 

3 

4 

303-19 

9034 

1382 

265 

51 

303 

91 

15 

0 

+ 3 

3 

5 

393-53 


1647 


50 

394 


15 





10681 


315 



106 


6 

-3 

3 

6 

500-34 

12643 

1962 

364 

49 

500 

127 

21 

1 

+ 1 +1 

3 

7 

626-77 


2326 


48 

627 


22 





14969 


412 



149 


7 

-3 

4 

8 

776-46 

17707 

2738 

459 

47 

776 

178 

29 

2 

+ 3 

5 

9 

953-53 

20904 

3197 

505 

46 

954 

209 

31 

6 

— 1 

5 

10 

1162-57 

24606 

3702 

550 

45 

1163 

246 

37 

5 


5 

11 

1408-63 

28858 

4252 



1409 

288 

42 




12 

1697-21 





1697 







This example illustrates that smoothness of differences of rounded 
values of a function is not a guarantee that these values give the best 
representation of that function. The adjustment of function values by 
differences cannot be depended on to ±1 unit in the last place; it is 
possible, as in this example, to make the differences over-smooth.f 

4.45. Direct evaluation of second differences 

It is sometimes convenient to be able to evaluate second differences 
directly from function values without the intermediate step of calculating 
first differences. This can be done on a machine as follows. 

Suppose first that the second differences of / are positive; is cal¬ 
culated from the formula 


8 % =/,-.+/(«- 2 /„ 

the terms being taken in this order; then/,- is set ready for the calculation 

8 %.. =/,+/, 

and so on. 

If the second differences are negative, this process will give them in 
complementary form; then it is more convenient to obtain 

f For a further discussion of checking by differences, see J. C. P. Miller, M.T.A.C. 
4 (1950), 3. 
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the terms being taken in this order so that when —has been obtained, 
fj+i already set for the calculation of 

If the function values are negative, their moduli are set, and the signs 
of the machine operations altered accordingly. 

This is a useful process for checking values of a function built up from 
second differences by summing the second differences to form the first 
differences, and then summing first differences to give the function 
values. The direct calculation of second differences provides a good 
check of these two successive summations. 

4.46. Building up from second differences 

A function can be built up directly from its second differences, without 
calculation of the first differences, by a process which is the converse of 
that of the previous section. If the function is positive, we have 

this is transferred to the setting levers, and used in the first step in forming 

If fj is negative, it is more convenient to form 

(-/y+i) = 2(-/,)-(-/, 

If this process of building up a function from its second differences is 
used, the method of the previous section should not be used for checking; 
the processes are too nearly alike for one to be a good check of the results 
of the other. 

One machine, the Brunsviga 20, has two facilities which are very con¬ 
venient for building up a function from its second differences; these are 
transfer from the accumulator to the setting levers, and an arrangement 
for clearing only the right-hand half of the accumulator, leaving the 
left-hand half unaffected. The latter feature islmown as ‘split clearance’, 
and has the effect of furnishing the machine with two registers. 

In the present application, the first differences are accumulated in the 
right-hand half of the accumulator (R.H. for short) and the function 
itself in the left-hand half (L.H.). Let be in R.H. and fj in L.H.; then 

is set on the setting levers (S.L.), and added into R.H., which then 
contains Bfj^.^. This is transferred to S.L., and the operation of clearing it 
from R.H. does not affect L.H.; it is added back into R.H. and also, after 
shifting the accumulator, into L.H.; R.H. now contains and L.H. 
contains fj+i- The accumulator is now shifted back, set, and the 
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process repeated. The only quantities needing setting are the second 
differences 8^fj. 

4.5. Differences and derivatives 

We have seen that functions defined by analytical formulae are ade¬ 
quately represented by tables only in ranges away from singularities and 
discontinuities, and that if a table is the only information we have about 
a function, we may regard the function represented by the table as being 
differentiable as many times as we require. We will therefore suppose 
that in any application of numerical methods to functions specified by a 
table, the function can be expanded in a Taylor series over the range with 
which we are concerned. 

Then we have 

and in general 

/t„ =/o±(»8a:)/„ + i(li8x)y'„±l(«8l)%"+..., (4.16) 


the remainder term being of order (8a;)'" if the series is cut off after m 
terms. We shall only derive a few relations directly from these expansions, 
as we shall shortly see a quicker and more effective way of deriving rela¬ 
tions of the kind we require in practical work. 

Substitution of series such as (4.15), (4.16) into the formulae giving 
differences in terms of function values gives a set of relations for 
differences in terms of derivatives; for example, if terms of order (8a;)® are 
included, 


sy,=/i-2/.+/-i 


2 


(S*)“^/'o+ ^ (S4:)</!,'+ i(8x)‘/J‘] + 0(Sx)» 


= ( 8 i)n/;+i'!( 8 »^)yo'+«!ii(s*)‘/?]+o( 8 *)*. (4.17) 


and similarly 

8% = A-Vi+O/o-V-i+A = (8*)*[/i'+i(84:)yj']+0(Sx)S. 

(4.18) 

From the symmetry of the coefficients in formula (4.17) it follows that 
even-order differences 82"/o involve only even-order derivatives at a;^. 
It follows from (4.17), (4.18) that 

Um[8%l{8x)^]=n, 

Sx-^0 

lim[S</<,/(8x)‘]=/i-, 

and similarly for higher orders of differences; thus finite difference 
ratios are closely allied to derivatives. But in using differences, the 

E 


6013 



60 


FINITE DIFFERENCES 


differences themselves are the quantities that enter into most formulae, 
rather than difference ratios. 

The relations (4,17), (4.18) and similar ones for higher-order differences 
can be regarded as equations for derivatives in terms of differences, and 
solved for these. A more important relation, however, is one between 
the second differences of f and its second derivative and its differences. 

From (4.17) applied to the function/" we have 

8% = {8xnf^+U^xW] + 0{Bxr, 

8% = {8xrf^-\-0{Sxr, 

{Sxfff = S%-i^S%+0{Sx)^, 

{Sx)‘ff = S%+Oiixf, 
and substitution into (4.17) gives 

8% = (Sx)TC+AS’/o'-a58‘/a+0(Sa:)». (4.19) 

Similarly, 

P^/o ~ i(/i — f-i) ~ (^^)|/o+ ^ ^ (^^)yo j + ^(^^)^» (4.20) 

and application of (4.17), (4.18) to /' gives 

8% = {8xnf^+u^xm]+o{8xr, 

8Yo = {8xrn+0{Sxr, 
and substitution in (4.20) gives 

p8/o = {8x)[fo+l8%-^B%] + 0{8xy. (4.21) 

As we shall see later (§ 6.3), the first two terms in the square bracket 
here give the formula usually known as ‘Simpson’s rule’ for numerical 
quadrature. 

For relations involving evew-order differences, and odd-order mean 
differences, the expansions (4.15), (4.16) in/and its derivatives a,tx = x^ 
are the most convenient. For corresponding relations involving odd-order 
differences and even-order mean differences, it is often more convenient 
to expand in terms off and its derivatives at x = x^. 

4.6. Finite difference operators 

A powerful method of obtaining formulae for interpolation, integra¬ 
tion, etc., in terms of finite differences is by means of finite difference 
operators. We have already recognized that the symbol 8 or A prefixed 
to a symbol representing a function can be regarded as representing an 
operation performed on that function. We will now extend this idea, and 
first define some further operators. 
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The operator E is defined by 

Ef{x) =f{x+hx), 

or shortly (4.22) 

This operator advances the argument from one value to the next of the 
finite difference table, and is sometimes called the ‘shift operator’ or 
‘forward shift operator’. Its inverse, written E-'^ or ^jE, the ‘backward 
shift operator’, steps the argument back from one value to the previous 
one in the difference table; that is 

(4.23) 

If D is the differential operator D = djdx, Taylor’s expansion can be 
written symbolically 

fi =fi^o+^^) = 

so that, formally, Ef^ = (4.24) 

for all functions/ for which the right-hand side is significant. A relation 
such as this, between results of different operations, which is independent 
of the function / operated on, is often written as a relation between the 
operators, without an operand explicitly indicated. We follow this usage, 
and, in accordance with it Avrite (4.24) as 

E = (4.25) 

Two operators of which we shall make considerable use are E^ and its 
inverse E~^. E^ is the operator which, applied twice to /q, gives 
independently of the particular form of the function /; that is to say, 
it is an operator such that for any operand /, 

EKE^h] =A = EU 

It is clear that an operator which advances the argument value by half 
the tabular interval satisfies this condition; that is 

E^f{x) = f{x-\-liSx), 

or (4.26) 

From Taylor’s series 

so that (4,26) is consistent with (4.25). 

The ‘forward difference operator’ A is defined by 

A/(a;) f{xf-8x)—f{x) = Ef{x)—f{x), 

or shortly A/o = {E—l)fo 

which, expressed as a relation between operators, is 

A = E—l. 


(4.27) 
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The ‘backward difference operator’ V is defined correspondingly by 

yf(^) =fix)—f{x—8x), 

V = 1-E-^ = (E-l)JE. (4.28) 


or 


The ‘central difference operator’ S is defined by 

8fix) =f{x+i8x)-f{x-^8x) = {Ei-E-i)f{x) 
which, expressed as a relation between operators, is 

S = Ei—E-i. (4.29) 

Another useful operator is the ‘averaging operator’ fi, defined by 
= mx+i8x)+fix-l8x)^ = i[Eif{x) + E-if{x)l 
i.e. /X = l{Ei-\-E-i). (4.30) 

These operators all have their inverses. We have already considered 
the operator inverse to E. The operator inverse to 8 is the ‘central sum 
operator’ a = 8“*^ defined by 

^fn ^fn—l~^fn—iy 

or CT = E^l{E —1). 

It should be noted that of, like an indefinite integral, is undetermined to 
the extent of an arbitrary additive constant. The operator inverse to 
fjL will be considered in § 5.2. 

These operators are all linear; that is to say if 0 is any one of them, 
and / and F are any two functions, then 

0{f+F) = Of+OF. 

The operators E, A, D, 8, and V are also commutative; that is, if 0^ 
and Og are two of these operations and / is any function, 

OiiOJ) = 0 ,( 0 ,/). 

a and 8 are not necessarily commutative, since o(8f) may differ from 
^(o/) by a constant, just as J {df/dx)dx may differ from / by a constant. 

Some useful relations may be obtained from (4.23) to (4.30). For 
example, from (4.29), gg ^ E—2+E-'^ 

(the operational form of 8% = /i—2/o+/_i), and from (4.30) 

= i(^;+2+^-i), 

whence 8^ = 4{ix^—l), 


or 


l + i8^ 


(4.31) 
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And if in (4.29), (4.30) we substitute for E from (4.25) we obtain the 
formal relations 8 = 2sinh^(Sa:)Z>, (4.32) 

[M = cosh|(8a;)Z). (4.33) 

Also we have 

(^;4-l)8 = Ei{Ei+E-i){Ei-E-i) 

= {Ei+E-i){E-l) = 2(^;—1)^. (4.34) 

Also 8” = [E-i{E—l)]^ = E-^^{E—l)^, 

so that 8 % = {E-1 )^E-i% = {E- 1 )V,--jn; 

expansion of (E—l)^ by the binomial theorem gives 


A:=0 

in agreement with (4.8). 


k^O 


We shall make considerable use of relations between operators, such as (4.32) and 
(4.33), which imply the use of Taylor’s series in the form 

e^^fix) =f{x + ^) (4.35) 

without a remainder term. However, in using the formulae we finally obtain by 
means of these relations, we shall in almost all cases retain only the first few terms, 
thereby making truncation errors in which the remainder term of the Taylor 
expansion can be considered as incorporated. In most cases an analysis of the 
truncation error and its relation to the remainder term in the Taylor expansion 
can be carried out by the method of § 6.8. But it will be as well to know for what 
kinds of functions this expansion can be used in the form (4.35). 

1. Polynomials: it is clearly exact for polynomials since the series terminates. 

2. Exponentials: if J{x) = then 

The ! 5 cries in tlie bracket converges for all values of ^ and a, and its value 
is e®^. Hence 

ei^J{x) = e®^e®"= = e“(='+^> =/(x+^), 

so that wc can apply (4.35) without restriction to exponentials in which the 
exponent is linear in x. 

3. Products of exponentials and polynomials: we will prove that if 

e(^z{x) = z{x+^), 

then e(^[xz(x)] = (x+C)z{x+^), 

so that if (4.35) can be applied to a function z(x), then it can be applied to 
xz{x); and so, by repetition of the argument, it can be applied to z{x) multi¬ 
plied by any polynomial. We have 

e^nxzix)] = [\+$D+^^^D^ + ..]xz{x) 

= xz + $[xDz + z] + ^ ^[xD^z + 2Dz] + ^ +... 
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Since z is assumed to be such that (4.35) is valid, the first square bracket is 
xz(x+^), and the second is iz{x-\-^). Hence altogether 

e(D[xz{x)] = {x+^)z{x+i). 

Thus (4.35) can be applied to products of exponentials (including circular 
functions) and polynomials; and, since the operator is linear, it can be extended 
to sums of products of exponentials and polynomials. 

4.7. Examples of the use of finite difference operators 

It is convenient, for brevity, to have a single symbol for the operator 


{8z)D; this will be written U, that is 

U - {8z)D. (4.36) 

Then the relations (4.25), (4.32), and (4.33) are 

E — , 8 = 2sinh^t7, p, = cosh^C/, (4.37) 

so that U = 2sinh“i|S (4.38) 

= [(sinh-4S)/i8]S. (4.39) 


Since 8/ = {8x)Df 0{8z)^ for any particular function / to which these 
relations between finite difference operators can be applied, it follows 
that in expanding these and other relations in powers of 8 or U, 8'^ or 
17"^ can be regarded as a quantity of order (8.r)''. 

4.71. Derivatives in terms of differences 

Taking the wth power of both sides of (4.39) we have 

f/" = [(sinh-i ^8)/|8]»8«. (4.40) 

Since (sinh“* 2)/2 is an even function of 2 , this expresses f/"= {8zy‘D"' 
in even powers of 8 if n is even, and in odd powers of 8 if w is odd. The 
available central differences 8''/_, of even order have integral values of j, 
whereas those of odd order have (integer+|) values of j. Hence this 
relation can be used to obtain expressions for even-ovder derivatives at 
tabular values, or odd-order derivatives half-way between tabular 

values. 

An alternative form is 

f/” = [p-i{(sinh-48)/^8}’']p8”. (4.41) 

Since the relation between p and 8 is p^ = 1 + |82, the operator in the 
square bracket is still an even function of 8, so that for odd values of n 
this expresses C/”/ in terms of odd-order mean differences p82"*+y, which 
are available at tabular values; hence this is the useful formula for odd- 
order derivatives at tabular values. 
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The expansions of (4.40) and (4.41) both for positive and for negative 
values of n can be carried out by taking the series for (sinh~^-^8)/-^S: 

(sinh-4S)/^8 = 

and raising it to the appropriate power; and in the case of (4.41) multi- 
plyiiig ^Iso by the expansion of = (1 + JS^)~^. General expansions 
for (C//8)” and [(C//S)”^/p] as far as 8^® for any value of n have been 
given by Bickley;t taken to terms in 8® that for {U/8)^ is 

/sinh~^|8\” 


/{7\” /sinh~^48\’ 

w “ri^j 


= 1 ^ §2 I 

24 5760 


35n3+462ri2+l528w 


2903040 


88 + 


(4.42) 


For positive n, the three cases of this formula which we shall need here 
are the cases n = 2, 4, and 6, namely, 


(C7/8)2 = [1_j1,82+^8«-5^88]+G(8x)8, (4.43) 

iU/8)* = [l-i82+^8«] + 0(8x)e, (4.44) 

(t7/8)6 = [l_J82]+0(8a:)«. (4.45) 

For odd positive powers of {U/8), the only important case is rr = 1, for 
(t7/f*5) = [l-J8^+i8«-,5ii8«+,|,8>]+0(Sa:)i». (4.4«) 


These formulae, which give powers of C/ = {8z)D in terms of 8, are 
operational forms of formulae for differentiation, since, applied to a 
function /, they give Z)”/ in terms of the differences of /. They have, 
however, other and more important applications as will be seen in the 
next section and subsequent chapters. 


4.72. Negative powers of (C//S) 

Other important relations are some involving negative powers of 
(C7/8). One way of obtaining these is by use of formula (4.42), or Bickley’s 
corresponding formula for iUI8y‘'ljx, with negative values of n, for which 
these formulae are also valid. For example, substitution of n = —2 in 
(4.42) gives 

(C7/8)-2 = [l+AS2-5ioS"+g&S8-3g|iooS8]+0(8x)i8. (4.47) 

Another procedure is first to express the operator in terms of U, expand 
as a series in U, and then substitute in terms of 8 from formulae (4.43) to 
(4.46); this only involves the use of formula (4.42) for positive values oin. 

I W. G. Bickley, Journ, Math, and Phys. 27 (1948), 183. 
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The main operators which involve inverse powers of U and for which 
we require expressions in terms of 8 are {8/U)^, (8/p.C7), and (p.8/C7). In 
terms of U they are 

(8/C7)2 = [(sinh|C7)/^C7]2 = 2(cosh U-1)IU^ (4.48) 

{SIfxU) = {ta,nhiUmU> ( 4 - 49 ) 

{fxS/U) = (sinh U)IU. (4.50) 

Expansion of (4.48) in powers of U gives 

(8/C7)»= l + lc7= + |,P‘+|c'*+0(Sx)» 

= l+A8*(C//8)*+si58‘(U/S)*+2!ik8'‘(P/S)*+0(8i)'- 

Substitution from formulae (4.43) to (4.45) then gives, to terms in 8®, 
(S/D)* = l+A,8*[l-ii8*+,J|S*]+iS*[I-JS*]+j^8«+0(8i)* 

= [l+A!8*-d55‘+5&5*]+0(Sa:)<' 
in agreement with (4.47). 

Similarly the following expansions can be obtained: 

(8/fiP) = [l-A8*+^8‘-iia8«+iaSiiS*]+C'(8*)“. ('‘•SI) 

((x8/D) = [H-J8*-,J,8Hrire8«-,a58»] + 0(8*)'«. (4.62) 

The latter of these can alternatively be obtained from the former by 
multiplying by = 14-J82. 

4.73. 82/ in terms off" and its differences 

We will use some of these relations between differential operators and 
finite difference operators to express 8^/^ in terms of fl and the central 
differences of/" at a: = x^. A first approximation is 

3 % = = U%-, 

to improve on this we must find an operator (^(8) such that 

8^fo = <f){8)U^fQ, (4.53) 

The operator (f>{8) required is therefore 

/(8) = (8/C7)2; 

its expansion in powers of 8 is given by (4.47) above, and substitution 
in (4.53) gives 

8*/, = (Sx)*[/;+j58y;--,i58y;+5&8»/;-,Si58»/;]+o(8:t)'* 


(compare formula (4.19) and its derivation in § 4.5). 


(4.54) 
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4.74. 8/j symmetrically in terms of /' and its differences at Xq 
and Xi 

By definition, 8/j = fi—fo = —l)/o> ^ first approximation 

= i(S^)(/o+/i) = i(Sa:)(£?+.l)-C>/o = i(-^+l)^/o’> (4.55) 

we want to obtain a more general relation of which this is the leading 
term. 

By a formula symmetrical in/' and its derivatives at the two ends of 
the tabular interval is meant one in which the coefficient a„ of each 
8’^f'i is the same as that of the corresponding 8 "/ q , so that these terms 
together give a contribution 

terms with w = 0 do in the first approximation (4.55). Hence we want 
a relation of the form 


8/j = (^-l)/o - l<f>{S){E-^l)UU, (4.56) 

to satisfy (4.56), /(8) must be given by 


= W+hu- 


(4.S7) 


This can be expressed in terms of U by substituting E = e^; this gives 

(f)(S) — (tanh |C7)/^t7. 

Alternatively, it follows from (4.34) that (4.57) can be written 

,^( 8 ) = SjfiU 

for which the expansion in powers of 8 is given by (4.51). Hence the 
required formula is 

/.-/»= j(8*)[/„+/;-^(sro+8r.)+a(8‘/'o+8r.)- 

-iffi5(S‘/»+8‘/.)]+0(8»)*. (4.58) 


This is an integration formula, for if f'(x) is given as a function of x it 

Xi 

enables the change in f{x), that is j f'{x) dx, to be evaluated in terms of 

a:o 

the values oif'{x) (see Chapter VI). 


4.75. /x8/o in terms of /' and its differences at a: = cco 
In this case we want to find an operator /(8) such that 

p8/o = (8a:)/(S)Z)/o. 

The appropriate /(8) is 


/(8) = p8/C7 = (sinh U)/U 
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of which the expansion in powers of 8 is given by (4.52). Hence 

= «/.-/-.) = (8*)t/i+i8y;-A8Vi+iA58«/i-aV/i] + 0(8x)‘i 

(4.59) 

(compare §4.5, formula (4.21)). This also is an integration formula, 
relating the change of / in an interval 28a; of x to the behaviour of its 
derivative in the neighbourhood of that interval. 
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INTERPOLATION 

5.1. Linear and non-linear interpolation 

Given a table of values of a function f{x) at a set of tabular values of x, 
usually, but not necessarily, equally spaced, we may require to deter¬ 
mine either the value of f{x) at an intermediate value of x, or the value 
of X for which f{x) has some specified value. The process for finding a 
result of this kind is called ‘interpolation’, and, when it is necessary to 
distinguish between them, the former is called ‘direct’ and the latter 
‘inverse’ interpolation. The distinction is not usually significant unless 
the tabular values of x are equally spaced; this case, however, is much 
the most usual. 

By ‘linear interpolation ’ is meant interpolation using the approxima¬ 
tion in which, for 0 < p < 1, we take 

/(cCo+pSa:) =/o+p8/i; (5.1) 

expressed graphically, this is interpolation along the chord joining the 
points and {Xi,fi). This process is valid so long as the tabular 

values of x are spaced closely enough; we will obtain later (§ 5.22) a 
quantitative criterion of what is ‘closely enough’ in this context. ‘Non¬ 
linear interpolation’ is interpolation in some form which takes account 
of the departure of the (x,/) curve from the chord between the points 
corresponding to neighbouring tabular values. 

There are two kinds of tables; first, those in which interpolation is 
required frequently enough to j ustify the use of intervals of the argument 
small enough for linear interpolation to be adequate; secondly, those in 
which interpolation will only be occasional, not frequent enough to 
justify the calculation and printing at small enough intervals for linear 
interpolation to be applicable. In the latter case, non-linear interpolation 
is necessary. But if non-linear interpolation were generally recognized 
as a standard process, the bulk of tables could be very greatly reduced. 
For example, a table of sinx to five decimals at intervals of 10° reads as 
given on p. 60. We shall see later that the formulae required for carrying 
out non-linear interpolation in this table are comparatively simple. We 
shall also see that for linear interpolation we require \8^f\ to be not 
greater than 2, so that at least 40 times the number of entries are required 
in order to obtain a table in which linear interpolation can be carried out. 
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sy 

X 

sinx 

s/ 

sy 


0° 

0 

17365 

0 

-528 

0 

10° 

•17365 

16837 

- 628 

-511 

+ 17 

20° 

•34202 

16798 

-1039 

-480 

31 

CO 

o 

o 

•50000 

14279 

-1519 

-435 

45 

o 

O 

•64279 

12325 

-1954 

-372 

63 

60° 

•76604 

9999 

-2326 

-307 

66 

60° 

•86603 

7366 

— 2633 

-221 

86 

o 

o 

•93969 

4612 

— 2854 

-139 

82 

o 

O 

00 

•98481 

1519 

-2993 

— 46 

94 

CD 

O 

o 

1 00000 


— 3038 


90 


The reduction in bulk achieved by the use of a large interval and non¬ 
linear interpolation is not important in the case of functions of a single 
real variable, but becomes important in connexion with functions of 
two variables (or of a complex variable), or functions of a variable and 
one or more parameters such as the Bessel functions Jn{x) and the 
Whittaker functions Wi- J^x). 

5.11. Linear interpolation 

The simplest form of interpolation is linear interpolation, or inter¬ 
polation by proportional parts, for which the interpolation formula is 
(6.1) above. 

In carrying out linear interpolation on a machine, there is a precaution 
against mistakes which should always be observed. Suppose first that 
S/j is positive. Having cleared the accumulator, set /q, add it into the 
accumulator, and clear the multiplier register. Then set S/j, add it in, 
and verify that the content of the accumulator is now f^. This checks that 
the right values of /q and S/j have been taken. If p has m decimals, the 
accumulator should first be shifted m places right, and /q and S/j then 
set on the extreme right of the setting levers or keyboard. 

For direct interpolation, pS/j is added to f^, if p is greater than i, the 
addition of S/j to /„ to check can be taken as the first step of this multi¬ 
plication; if p is less than |, 8/j should be subtracted froin/j to restore 
/q before doing the multiplication. 

For inverse interpolation, the given value of/is built up in the accumu¬ 
lator, and the fraction p of the interval length required to give this value 
of / is read on the multiplier register. 
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If 8/j is negative, |S/j| should be set, and operations of addition and 
subtraction are interchanged; otherwise the procedure is the same. 

In some tables, particularly elementary ones, a sequence of function 
values is given on a single hne (for example, log I’OO to 1’09 on a line of 
a four-figure table of logarithms) with proportional parts of the mean 
first difference at the end of the line. Use of these proportional parts 
of the mean difference does not usually give the best interpolated 
value, and should not be used indiscriminately except in contexts in 
which an error of 2 or 3 units in the last figure is imimportant. The 
following example is taken from a table of logarithms to five places in 
which, for x — 1*0 to 2*0, different sets of proportional parts of mean 
differences are given for every five entries: 

* 1-05 1 00 107 108 1 09 I 1 2 3 . . 6 ... 9 

log I -02119 -02531 -02938 -03342 -03743 | 40 81 121 . . 202 . . 364 

The last five columns are the proportional parts of the mean differ¬ 
ence. Using the actual difference between the first two entries, we get 
log 1-055 = 0-02325, whereas using the proportional parts of mean differ¬ 
ences we get 0-02321, a difference of four units in the fifth decimal. 

For the best linear interpolation, proportional parts should be taken 
of the actual difference between successive tabular values. Tables of 
proportional parts for this purpose are given in most good modem books 
of tables. 

If several functions are tabulated in parallel columns, at such an 
interval that hnear interpolation can be used on each of them, then 
linear interpolation can be used between two columns. For example, 
a table of sin a: and cos a: against a:, in parallel columns, is also a table 
of (1—against y, and can be used as such without reference to the 
X column at all. Since the values of both functions f{x) and g{x) are 
subject to roimding error, the possible error in the interpolated value 
is rather greater than if/(a:) were tabulated at exact values of g{x). 

5.2. Non-linear interpolation 

In considering non-linear interpolation, it will be supposed for the 
present that the tabular values of the argument are equally spaced.f 
Interpolation with unequally spaced values of the argument will be 
considered in § 5.7, 

5.21. Half-way interpolation 

One particular case of non-linear interpolation is so much simpler 
than the general case, and so useful, that it will be considered separately 

f For a fuller treatment of non-linear interpolation, see L. Fox, The Use and Con- 
atruction of Mathematical Tables (H.M.S.O., 1956). 
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first. This is interpolation for a value of x half-way between tabular 
values. 

To get a formula for this, we want to express f{xQ-\-^8x), which can 
be expressed as E^f^, symmetrically in /q, and the differences of / at 
Xq and Xi. Now /{Xq-^-^Sx) = E^f^, so we want to find an operator /( 8 ) 
such that E% = i,{Sm+E)U 

This operator is given by 

so that = 2Eil(l-\-E) = 1/cosh^C/ = l/(l + i82)i, (5.2) 

/i =/(a:o+W = (l + iS^)-^[^(/o+/i)] 

= ilfo +/l ~ 5 (^^0 + ) +l| 8 (^Vo + ) “ 

-iol4(S%+8yi)]+0(8.T)«. (5.3) 

It will be noted that the operator /( 8 ) given by (5.2) is the operator 
inverse to the averaging operator p. Indeed, the relation (5.3) could be 
obtained as follows. The definition of the operator /x is /x/j = |(/o+/i), 
and it follows that the inverse operator /x~i is an operator such that 

fi = l/^“M/o+/i)- (^•'^) 

But /j — f{xQ-j-^8x) and /x^ = 1 - 4 - 58 ^, so that (5.4) is just (5.2) in a 
different form. 

Formula (5.3), perhaps taken to higher orders of differences, is useful 

in a preliminary breaking down of the interval of a table of a function 

evaluated at a large interval, before carrying out a sub tabulation. The 

coefficients are easy to calculate and to check if more are required than 

are given in (5.3).f If (— )%y is the coefficient of (S^^o+S^^fi) in the square 

bracket in (5.3) then , , ,v/o/-i i\ 

V > = ( 2 ^_|.l)/ 8 (^ + l) (5.5) 

and the coefficients can most conveniently be calculated by continued 
multiplication by the successive ratios (5.5). A check is given by the 
relations = 2/V3 = M5470. 2 = V2 = 1*41421. 

It is interesting to examine the result of applying (5.3) to a trble of cosxov sin.T 
at a large interval sueh as 60° or 90°. The ratio (5.5) tends to the value J for large^. 
Henee provided \ < 4 for large y, the infinite series of whieh (5.3) gives 

the first few terms formally eon verges. 

Now if f(x) = B eos(x-\-^)y then S^fj = — 2(1 —eosS.c)A, so that if Sx = 

I — 2. Thus by use of the series (5.3) we ean interpolate eosa; and sin a;, 
not only approximately but, by taking enough terms, to any aeeuraey we require, 
from the tabular values 


X 

cosx 
sin a; 


0 

1 

0 


iTT 

0 

1 


rr 

-1 

0 


0 

-1 


277 

I 

0 


and the eondition of periodieity. 

t See also Interpolation and Allied Tables (1956 edition) § C6, p. 58. 
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It is even possible to interpolate accurately from a table at intervals of fw: 


X 

0 


hr 

277 

cos a: 

1 

-I 

-I 

1 

sinx 

0 


-iV3 

0 


extended by using the condition of periodicity. 

5.22. Newton’s forward-difference formula 

Of the formulae for non-linear interpolation for a general value of the 
fraction p of the interval length, the simplest to derive is one in terms 
of forward differences. Its practical value is, however, limited. 
Taylor’s series can be written, in terms of operators, 
f{x^-\-phx) = = EPU 

Also E — 1 A, and expansion of (1+AlJ’ by the binomial theorem gives 
f{x^+p^x) = |^l+pA-|-lp(p-l)A2+...J/„ 

= /o+;^A/o+^p(p—l)A2/o+^p(p—l)(p—2)AVo+... (5-6) 

which is usually known as Newton’s formula. It uses values of the 
differences on an inclined line in a difference table: 


X 

^0 


a-. 


/ 

/o 



It is unsatisfactory if differences beyond the second have to be taken 
into account, as the differences of a function / depend primarily on the 
behaviour of the function in the neighbourhood of the value of x on 
which they are centred, so that the higher-order differences involved in 
this formula are less and less closely related to the behaviour of / in the 
interval in which interpolation is being carried out. 

Its practical use is restricted to interpolation near the boundaries of 
a table, and this is rare because unless / or one of its derivatives has a 
singularity atx — x^, there should usually be little difficulty in extending 
the table backwards a few intervals from x = Xq, whereas if the boundary 
of the table results from / being infinite at a; = (for example f{x) = 
cot a; at a: = 0) or undefined for a; < a;^ (for example/(a;) = a;^ at a; = 0), 
this situation is iisually associated with an infinite derivative/'(aj^), in 
which case the Taylor series expansion on which Newton’s formula is 
based is invalid. 
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There are various other interpolation formulae, which can all be 
derived from Newton’s by substitution for the forward differences 
A”/o in terms of differences more representative of the behaviour of f{x) 
in the interval through which the interpolation is being carried out. It 
is difficult, however, to obtain the form of the general term by such a 
derivation, and it is better to derive these other interpolation formulae 
independently. Of the various formulae Comrie writesf ‘only three are 
found in good modern practice, namely those associated with the names 
of Bessel and Everett, each of which is a simple transformation of the 
other, and that of Lagrange’. The present treatment will be restricted 
to these three. 

From Newton’s (or Bessel’s) formula it is possible to deduce the con¬ 
ditions in which linear interpolation gives a sufficiently accurate result. 
The greatest numerical value of the coefficient of the second difference 
in formula (5.6) is It is best to keep the contribution from this term 
to the interpolated value less than 0-3 in the last figure; if it were 
greater it should be included as it might affect the rounding off of the 
final result. Hence linear interpolation should not be used if second 
differences are greater than 2 unless errors up to 2 units in the last 
place of the interpolated value can be tolerated. 

Occasionally the contribution from the second differences to the inter¬ 
polated value is negligible when those from higher orders of differences 
are not; an example is provided by the function tabulated 

at unit intervals of x and interpolated between a: = 0 and 1. To avoid 
this situation it is only necessary to see that not only the second differ¬ 
ences used in the interpolation formula, but also a number of neighbour¬ 
ing values, are not greater than 2 in the last figure. 


5.3. Some expansions 

For the purpose of deriving interpolation formulae in central differ¬ 
ences,$ we shall require some expansions, namely those of sinhjSH, 
(coshj3?7)/cosh and (sinhj3C7)/sinh U in terms of 8 = 2sinh^?7, for 
non-integral j8. These could be written down from the similar expressions 
for circular functions of a numerical variable ;§ but their derivations 
will be given here for completeness. 

For the purposes of this section, let u stand for an ordinary numerical 


variable, and let 


z — 2 sinh \u and y = cosh j8m. 


(5.7) 


f Chambers^8 6-Figure Tables^ vol. 2 (1949), Introduction, p. xxvii. 

X The treatment of this and the following section follows that of J. G. L. Michel, 
Journ» Inst, of Actuaries^ 72 (1946), 470. 

§ e.g. T. J. I’A. Bromwich, Theory of Infinite Series (MacmiUan, 2nd ed. 1926) § 68. 
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Consider first the expansion of 1 / as a power series in z. We shall obtain 
this by forming the differential equation for y in terms of z as independent 
variable, then differentiating n times and putting 2 = 0 ; this will give 
recurrence relations for the derivatives d^yjdz'^ at z = 0 , from which 
their values, and so the required series, can be written down. 

Since y = cosh ^u, it satisfies the equation 



(5.8) 


and since z = 2 sinh ^u, it follows that 

fiz 

— = cosh^zi, (5.9) 


so that g = (cosh i»)|[(cosh i«)|]. 

On differentiating this out, substituting for sinh^zz from (5.7) and for 
dujdz from (5.9), we obtain 

(i+}^=)g+i4 = «'' («■>«) 

and then, on differentiating n times with respect to z and putting z = 0, 

^n+2)(0) = (^2_J^2)y(n)(0). (5.11) 

Also, for small u, z = and so 

y= l + 0{u^)= l + <9(z2) 

and hence z/(0) = 1, y'{0) = 0. Hence, from (5.11), for the odd deriva- 
tives ^2n+i)(o) = 0 

and for the even derivatives 

^"(0) = y‘^(0) = W2-1), z/^(0) = ^2(^2-l)(j32-4)..., 

and in general, in terms of the functions i/r„, introduced in § 4.41, 

y(2n)(0) = (^*12) 

Hence y = coshjSzz = 1+^3 2 'A2n+i(^)2^”'^V(2w+2)!. 

n 

Differentiation with respect to z then gives 

;S(8inh/3a)f^ = ftzj, iA,„+,(/S)2“”/(2™+I)!. 

CiZ n 

But from (5.9) 

zdz/du = (2 sinh |zz) cosh |zz = sinhz^; 

F 


801S 
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hence 

= I (5-18) 

Sinn u n 

= ;8[i + L(^»-1)z* + ^(^'-1)(^'-<‘)^‘+-]- (5-14) 

To obtain corresponding expressions for sinhjSw and coshjSM/cosh take 
2 = 2 sinh iu as before, and y = sinh jStt. This also satisfies equation (5.8), and the 
above argument applies as far as the recurrence relation (5.11). Now, however, 
y = ^z+0(z^) for small z, .so that i/(0) = 0, y'(0) = j9. Hence y<*")(0) = 0, and 

2/'(0)=i9, 2/"'{0) =iS(i3^-i), 2/^(0) =^(^*-i)(^*-f), 

and in general, in terms of the functions of § 4.41 (p. 40), 

2 /'^"+‘>( 0 ) = P>p2n(P) 

(compare (5.12) for the expansion of co.shj3M). Hence 

y = sinhjSw = jS 2^2„(i3)z""+V{2n+l)! (5-15) 

n 


The expansion of (cosh j9w)/(cosh can be obtained by differentiating (5.15) 
with respect to 2 :. On the left-hand side this gives j9(coshjStt) {dujdz). But from 
(5.9) this is just j9(cosh j3w)/(cosh Hence 

cosh j3w 


cosh 




(5.17) 


5.4. Everett’s interpolation formula 

The simplest central-difference interpolation formula to obtain is that 
known as Everett’s. This expresses the interpolated value of/in terms 
of the values of / and of its even-order differences only, at the beginning 
and end of the interval in which the interpolation is being carried out; 
that is, it is of the form 

/(a:o+pSa:) = (l-p)/o+^)/l^-^^2(^>)S!/o+■^2(^’)S!/'l+ 

' (5.18) 


The coefficients in this interpolation formulaf are usually known as 
‘Everett interpolation coefficients’; they are functions of the fraction p 
of the interval length Sx for which the interpolation is being carried out. 

To obtain a formula of this kind we must find operators <f>o(S), ^i(S) 
which involve only even powers of 8 and which are such that 

/(xo+pSx) = /o(S)/o+/i(S)/i. (5.19) 


I The notation p for the fraction of the interval Sx for which interpolation is carried 
out, q = 1 — p, and ^^id f®^ coelficients in the Everett formula, here used is 
adopted to conform to that of Interpolation and Allied Tables (H.M. Nautical Almanae 
Office), 1956, and L. Fox *The Use and Construetion of Mathematical Tables’ (H.M.S.O., 
li)56). The coefficients here written and are also written and 
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Now/(a:o+p8a:) = eP^f^, and/^ = e^f^, so (5.19), expressed as a rela¬ 
tion between operators, becomes 

= 4>oi^)+U8)e^- 

Since <i>o{S), (f)^{8) are to be even functions of 8, and so of U, it follows 
that they do not change on replacing U hy —U; hence 

e-pu = 

and solution for <f>o{8), 4>i{8) then gives 

^i(S) = {eP^ — e~P^)l{e^—e~^) = sinhpC//sinh U, 

^o(^) — sinh(l—p)C7/sinh U — sinh^'^Z/sinh C7, 
where, for convenience later, q has been written for (1—p). 

We require expressions for these operators in terms of 8. These could 
be obtained by expanding in powers of JJ and then substituting for JJ 
in terms of 8 from formulae (4.43) to (4.45); but the form of the general 
term is most easily obtained by the formal substitution of U for u, 
8 for 2 , and p or q = 1-p for in (5.13) or (5.14). This, followed by 
substitution of the results into (5.19), gives 

/(x„+pSx) = 2 ( 2 ^^, [(<' 2 .-h( 1 -:P) 8 ""/o+( 62 „«(p)S'‘”/,] (5.20) 

+;'[/i + ^(p“-l)Sy,+ 4(p2-l)(;)»-4)SV, + ...l (5.21) 

which is Everett’s interpolation formula. The leading terms, qfo+pfi, 
give the value fo-\-p8fi obtained by linear interpolation, expressed in a 
form consistent with the other terms of the formula. 

Comparison of (5.20) with (5.18) gives the following general expres¬ 
sions for the coefficients in (5.18); 

5.41. Bessel’s interpolation formula 

Bessel’s interpolation formula expresses the interpolated value 
f{xo 8x) in terms of mean differences of even-order p.8^”/j and odd-order 
differences 8^"+yj, centred on the middle of the interval in which inter¬ 
polation is being carried out. For practical work it is most convenient 
to have the contribution from the even-order differences expressed in 
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terms of the sum of the values at the beginning and end of 

the interval. Thus this formula is of the general formf 

nx„+pix) = i{/<,+A)+{j)-i)8/,+ B,{p){8%+8%)+ 

+-B3(p)8y,+ B,{y){8%+8‘/i)+iJa3))8y,+ .... (5.23) 

The coefficients BJ^p) of successive orders of differences in this 
formula are called ‘Bessel interpolation coefficients’, or simply ‘Bessel 
coefficients’ when there is no danger of confusion with the other meaning 
of this term. The first two terms give the value /o+p8/i obtained by 
linear interpolation, expressed in a form consistent with the other terms 
of the series. 

A formula of this kind can be derived quite easily from Everett’s 
formida (5.18). Consider the pairs of terms involving and 5^”/i 
in Everett’s formula. These can be written 

= i[®2„(3>)+-f’2.(3>)](8'=”/o+8'‘”/i)+K-f’2„(j))-«2„(j')]8“-+Vl. 
which is of the form of the contributions from and in Bessel’s 

formula. Comparison with (5.23) and use of the formulae (5.22) for the 
Everett coefficients gives 

Bzniv) — i[-^2n(p)+-^2n(l>)] 

^ (271 I P')^2n+\^Py\ 

and 

^2n+\{P) ~ W.^2n{P) ■^2n.(P)] = 2 (2?l-t-l)! ^2n+l(l P)Y> 

and on substitution from (4.13), (4.14) these become 

Binip) = 2 i) 

Bin+xip) = 1) ! i) 

The first few functions B^{p) are 

Biip) =p(p-l)/2.2! B^{p) = p(p-i)(p-l)/3! 

Bi(p) = (p+l)p(p—l)(p—2)/2.4! 

BM = (p+l)p(p—|)(p—l)(p—2)/5! 

t In this formula, B^(p) is written for the coefficient of + not for the co¬ 
efficient of fiSVj, and similarly for the higher even orders of differences. This usage 
follows that adopted by Comrie (Chambers's 6-Figure Tables, 1949, vol. 2). In some 
earlier work and tabulation of coefficients in Bessel’s formulae B" or B** has been used 
for the coefficient of 
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Bessel s formula can alternatively be derived directly without using Everett’s 
formula; the following is a summary of this derivation. 

Expressed in terms of operators, formula (5.23) can be written 

where is an even function of 8 and <f> 2 {B) an odd function. Thus the operators 
and <f> 2 iB) must satisfy 

E^-i = <f,,{8){Ei+E-i)+4>S), 

that is = <^i(S).(2cosh ii7)+^j(8). 

But = cosh(p-i)i7+sinh(p-J)i7, 

of which the first term is an even function of U and so of 8, and the second is an 
odd fimction. Hence we obtain a formula of the kind sought by taking 

^i{S) = cosh(p — J) 17/2 cosh ^17, — sinh(p—J)?7. 

The expansions of these in powers of 8 = 2sinh^l7 can be written down from 
(6.16), (5.17) by making the formal substitutions of U for u, 8 for z, and {p — ^) for j8. 

5.42. Use of Bessel’s and Everett’s formulae 

Bessel’s formula to second differences, namely, 

f{xo-hp8x) =/o+^>8/i+B2(2))(82/o+8yi). (5.25) 

or to third differences, with second differences modified as explained 
below, is generally the most useful formula for non-linear interpolation, 
unless so large a number of figures is required, or the spacing 8a; is so 
large, that fourth and perhaps higher-order differences have to be taken 
into account. Then Everett’s formula is probably more convenient, 
especially when using tables in which only differences of even order are 
tabulated. 

The coefficient is always negative. A critical table to three 

decimals is given in Comrie and Milne-Thompson’s Standard 4-Figure 
Tables and one to four decimals in Interpolation and Allied Tables (1956), 
where B^{p) and B^{p) are also tabulated; B.^{p) and B^{p) are also 
tabulated in Chambers's 6-Figure Tables, vol. 2 (1949). For other tables 
of coefficients in this (and other) interpolation formulae, reference should 
be made to the Index of Mathematical Tables. 

In Bessel’s interpolation formula, the coefficients of the odd-order 
differences are all zero atp = ^ as well as atp = 0 and 1; this is an advan¬ 
tage over most other interpolation formulae which involve all orders 
of differences. The greatest value of | B^{p) \ is about 0-008, so the 
contribution from 8^/ to the interpolated value is less than 0-5 in the 
least significant figure if |8y| is less than 60. 

Further, the contribution from the second and fourth differences 
together is 

B,(p)[(s%+sy,)+A(j)+i)(p-2)(8%+sy,)] 
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and does not vary greatly over the range of p, from 

0 to 1, over which this formula will be used; its maximum value is 0’1875 
at ^ ^ and it is greater than 0*180 over half this range ofp; its smallest 
value is 0*1667 a,t p = Q and 1, where B^{p), by which it is multiplied, 
is zero. Hence a good approximation to the contribution from to 
the interpolated value can be made by subtracting a constant multiple 
of Sy from each 8% and applying Bessel’s formula, correct to second 
or third differences only, with the second differences so modified. If 

= (5.26) 


and use B 2 {p){ 8 mfo~^^mfi) such a formula, the residual contribution 
from 8 ‘/is ij.(j>-l)[C'+A(j.+ I)(il-2)];xSy,. (5.27) 


The best value of C is that which makes the extreme values of the 
coefficient here equal and opposite, and is C' = 0*184; the greatest value 
of the coefficient of in (5.27) is then 0*00045, whereas the greatest 
value of \B^{p)\ is 0*0117. The residual contribution from Sy is less 
than 0*5 in the least significant figure if Sy is less than 1100. 

Quantities 8 ^/^-given by (5.26) with (7 = 0*184 are called ‘modified 
second differences’ and this inclusion of a constant multiple of the 
fourth differences in modified second differences is called ‘throwback’ 
of the fourth differences to the second. It is due to L. J. Comrie, and 
is a valuable device for simplifying practical interpolation, particularly 
inverse interpolation and subtabulation. 

In Everett’s formula the coefficient of 8 ^”/^, is the same function 
of ( 1 —p) as i^ 2 n is ofp, so that in tables of interpolation coefficients the 
number of separate functions which have to be tabulated for Everett’s 
formula is only about half as many as for formulae involving all orders of 
differences. Also in tables of the function /to be interpolated, only even- 
order differences'need be given. Tables of Everett coefficients are given 
in Interpolation and Allied Tables and in Chambers's 6-Figure Tables, 
vol. 2 (1949); tables at the close interval 0*0001 inp have been published 
by the Mathematisch Centrum, Amsterdam.f 

The ‘throwback’ can be used with Everett’s formula as with Bessel’s. 
The contribution from and together in Everett’s formula is 




the coefficient —^(p^—4) varies from 0*15 to 0*20 over the range p = 0 


J Tables of Everett's Interpolation Coefficients by E. W. Dijkstra and A. van Wijn- 
gaarden (Amsterdam, 1955). 
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to 1, and is multiplied by a zero factor at both ends of the range. If 
the same modified second differences are used, namely, 

8^/=S2/-0-1848V, 
the residual contribution from 8 “*/^ is 

l7>b'-l)[0-184+Jg(p2_4)]SV,; 

the greatest value of this coefficient is about 0-0008, so that this con¬ 
tribution is less than 0-3 in the last figure if is less than about 400. 
Similarly for the contribution from 

If fourth differences are too large to be treated by means of the throw¬ 
back, Everett’s formula can be taken as far as the 8y terms, and the 
sixth differences thrown back to the fourth differences.f If eighth 
differences are appreciable, very effective use can be made of a joint 
throwback of the sixth and eighth differences to the second and fourth 
differences.f 

5.43. Practical details in non-linear interpolation 

In using Bessel’s or Everett’s formulae, values of the coefficients can 
either be calculated as required or taken from tables. In the latter case 
the interpolation will have to be done in two stages if the number of 
decimals in p is greater than that in the argument of the tables of inter¬ 
polation coefficients. One method of dealing with this situation is to 
carry out a subsidiary interpolation in the tables of inteq^olation co¬ 
efficients themselves. But it is generally better to carry out a small 
subtabulation of the function f{z) using only tabular values of the inter¬ 
polation coefficients. For example, if f{z) is tabulated at intervals 
8 z — 0-1 and its value is wanted for z = 0-854377, and available tables 
of the interpolation coefficients have the argument p at intervals 8 p = 
0-001, the values of f{z) for z = 0-8541(-0001)0-8545 can be obtained 
without interpolation in the tables of the interpolation coefficients, and 
interpolation in this small table o{f{z) will then give the result required; 
linear interpolation will often be adequate at this stage. 

In carrying out a non-linear interpolation, it is advisable to carry 
one guarding figure to avoid accumulation of rounding errors from the 
various contributions to the interpolated value. For a similar reason, 
a guarding figure should be kept in the subtabulation mentioned in 
the previous paragraph. Also it is advisable to retain contributions 

f For these and other developments of the idea of the throwback, see Chambers's 
6'Figure Tables, vol. 2 (1949), p. 533. 
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greater than 0*2 in the least significant digit from the higher orders of 
differences. 

On this basis: 

In Bessel’s formula, with throwback of fourth differences to second: 

Sy can be neglected if less than 15 
Sy can be neglected if less than 500. 

In Everett’s formula, with throwback of fourth differences to second: 
Sy can be neglected if less than 250. 

Examples : 

(a) Given the following values, to find/(f): 


X 


8/ 

sy 

sy 

(syo+syi) 

0-60 

1-66667 

-5377 




•62 

•61290 

— 6040 

337 

-32 


•64 

•66250 

— 4735 

305 

-26 


•66 

•51515 

— 4456 

279 

-25 

+ 533 

•68 

•47059 

-4202 

254 

-20 


•70 

•42857 

-3968 

234 



0-72 

1-38889 






Here {hx) = 0*02, x == f = 0 - 66 +f( 8 x), p — The contribution from the third 
difference is just worth taking into account, but the fourth-difference contri¬ 
bution is negligible, even without using the throwback. The value of B^ip) is 
\p{p—\) — — 3 ^, and that of B^{p) is ff) = H-iiy. Hence we have 

/o = 1-51515 

phf^ =. - 0*014853 
B^{p)[h% + h^f{\ = -0-00029e 
Bz{p)h^h = - 0-000002 

l-49999j,, 
or rounded off, T50000. 

The guarding figure is written here as a suffix; this is a convenient convention. 
Notes: (i) The point of expre^ing the second-difference contribution in the form 
+ rather than { 2 jB 2 (p)}jLiS^/^, is clear from this example. If the 
quantity 8 %+ 8^1 is odd, then in dividing by 2 to obtain we would either 

have to round off or keep an extra figure, and this is avoided by incorporating the 
division by 2 in the factor B 2 {p) = — 1 ) by which this quantity is multiplied. 

(ii) The function J{x) here is 1 /a;, the tabular values being rounded off to five 
decimals, so that the correct value of/(f) is T 5 exactly. 
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(b) Given the following values, to find/(f): 


X 

f{x) 

8 / 

8 y 

8 y 

sy 

-0-1848y 

0-50 

2-00000 








— 

18182 






-55 

1-81818 


3031 






— 

15151 


-701 




-60 

-66667 


2330 


203 

-37 

2293 


— 

12821 


-498 




-65 

•53846 


1832 


131 

-24 

1808 


— 

10989 


-367 




-70 

•42857 


1465 


93 

-17 

1448 


— 

9524 


— 274 




-75 

•33333 


1191 


63 

-12 

1179 


— 

8333 


-211 




•80 

•25000 


980 






— 

7353 






0-85 

1-17647 







Her© (Sx) = 0*05, x 

= f = 

0-65 + ^( 8 x), so p 

= h 

? = f- 





3256 


(i) By Bessel’s formula 
■BaC??) = ip{p — l) = —1^ 

BsiP) = lP(p—^){P — i) = +T62 

/o = 1-53846 

pSfi = — 0-03663o 

Ba(p)(8^/o + 8^A) = -0-00180, 
Bt{p)8^fi = - 0 - 00002 , 
1-49999, 


(ii) By Everett’s formula 

e.Sp) = -k(i-g*) = 

FiiP) = — iMl—= —A 

/o = 1-53846 

p8fi = — 0-03663o 
E^{p)Kfo = - 0 - 00111 , 
F^{p)hlf, = - 0-00071, 
l-49999„ 


Rounded off to five decimals = 1-50000 


Rounded off to five decimals 

= 1-50000 


In this example modified second differences have been used, and the residual 
contributions from fourth differences are negligible. If modified second differences 
had not been used, it would have been necessary to include the fourth difference 
terms in each case. 


(iii) By preliminary snbtabnlation: 
p -30 

-32 

•34 

•36 

Eiip) = k(i-?*) 

-05950 

-06093 

•06208 

•06298 

F^(p) = ip(l-y) 

-04550 

•04787 

•05012 

•05222 

X 

-665 

•666 

•667 

•668 

fo 

1-53846 

1-53846 

1-53846 

1-53846 

pSfi 

— 3296, 

— 3516, 

- 3736, 

- 3956, 

f3(l-9*)8^/o 

107, 

- 110 , 

- 112 , 

113, 

ip(l-pWi 

65, 

69, 

72, 

75, 

f 

1-603763 

l-60150o 

1-49924, 

1-497005 

Differences at the smaller interval | 

225, 

O 7 

225, - 

0 , 

■224, 


Linear interpolation between the subtabulated values is now adequate; p = f, so 
/(f) = l-50160o-f(226i) 

= 1-50000 on roimding off to five decimals. 
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Note: Since the interval length has been reduced by a factor 50, second differences 
are reduced by a factor 2500 from those of the original table, so their values would 
be expected to be about Og in the fifth decimal, and are certainly negligible for inter¬ 
polation purposes. It is then only necessary to calculate two values of/(for example 
those for x = 0'666 and 0'667); but four values have been calculated to give a 
partial check. A thorough check of an isolated interpolation is difficult to achieve, 
but a good check is provided by carrying out the interpolation between two sets 
of values of the original function at different intervals. 

5.5. Lagrange’s formula 

The interpolation formulae so far given have expressed the inter¬ 
polated value of/(a:) in terms of contributions from its various orders of 
differences. An alternative type of formula expresses the interpolated 
value oif{x) as a sum of multiples of the values of the function / itself, 
with coefficients which are functions of the fraction p of the interval 
length for which the interpolation is required, thus: 

fi^o+P 8 ^) = 1 hiP)fy 

3 

An interpolation formula of this type is called a ‘Lagrange interpolation 
formula’, and the coefficients Lj{p) are knowm as ‘Lagrange interpolation 
coefficients’. There are several formulae of this type, with different 
numbers of terms taken in the sum in (5.28), and correspondingly with 
different sets of coefficients. 

A formula using n function values is usually known as an ‘w-point’ 
formula; n is usually taken as even, and an equal number of points 
taken on each side of the interval in which interpolation is to be carried 
out. Such formulae can be obtained by expressing the finite differences 
in Bessel’s or Everett’s formula in terms of function values by formula 
(4.8) and collecting terms involving the same function value. But they 
are more conveniently obtained as special cases of a formula, which will 
be derived in § 5.7, for interpolation of a function given at unequal 
intervals of the argument. An n-point formula is based on the approxima¬ 
tion to/by a polynomial of degree {n —1) through n successive values 
of the function, interpolation being carried out by evaluating this 
polynomial at the value of x for which the interpolation is required. For 
even values of w, use of such a formula is equivalent to the use of Bessel’s 
formula to {n — l)th differences or of Everett’s to {n —2)th differences, 
without throwback. 

Tables of Lagrange coefficients for 4-point and 6-point interpolation, 
for p = 0(0-01)l-00, are given in Chambers's 6-Figure Tables-, for other 
tabulations the Index of Mathematical Tables should be consulted. 
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The advantage of Lagrangian coefficients formulae, if it is an advan¬ 
tage, is that they can be used directly on tables in which no differences 
are given. On the other hand, they have several disadvantages, as 
pointed out by Comrie:t 

(i) They provide no check that the function values used in them have 
been taken correctly, whereas the differences used in a difference 
formula also provide a check on the function values used; 

(ii) A single calculation of an interpolated value provides no indica¬ 
tion whether the degree of the polynomial used is inadequate, 
adequate, or excessive; 

(iii) At least one of the coefficients Lj{p) is greater than 0*5; if there¬ 
fore an interpolation is required for a value of p which is not a 
tabular value in the table of Lagrangian interpolation coefficients, 
interpolation in these tables is required to the same number of 
significant figures as that required for the interpolation of/itself; 

(iv) Use of Lagrangian formulae does not lend itself to an easy process 
for inverse interpolation. 

To these may be added: 

(v) They do not provide the facilities for improving the accuracy of 
interpolation without complicating the formulae, such a.s are 
provided by interpolation formulae in terms of differences by 
use of the ‘throwback’. The Lagrangian formulae are based on 
the approximation to the function to be interpolated by a poly¬ 
nomial of the nih. degree through (n-f-1) points. But consider 
the significance of the use of the throwback from fourth to second 
differences and subsequent use of Everett’s formulae to second 
differences. The fact that this modification of the second differ¬ 
ences improves the accuracy of the interpolation (and moreover 
by a factor of about 10, not only by a small amount) means that 
for interpolation between /q and /j the best cubic is not the cubic 
through /_!, /o, /i, /a which is the one used in the four-point 
Lagrange formula. 

Certainly there is a formula of Lagrangian type corresponding 
to the Everett formula to second differences, used with modified 
second differences; but this is no simpler than a six-point Lagran¬ 
gian formula based on the use of a quintic polynomial. The 
corresponding formula using differences is Everett’s to fourth 
differences, and if in this the joint throwback of sixth and eighth 

t Chambers's 6-Figure Tables, vol. 2 (1949), Introduction, p. xxix. 
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differences to second and fourth is used, a very substantial 
improvement in the accuracy, compared with that of a six-point 
Lagrangian interpolation, is achieved. 

Comric’s commentf is that he ‘has to admit that his experience has 
not made him partial to blind Lagrangian interpolation, except when 
special circumstances point very definitely to it’. 

5.51. Special interpolation methods for particular functions 

For some particular functions, special interpolation methods may be 
(— more convenient than the use of the Bessel or Everett formulae. For 
example, for the exponential function it may be most convenient to use 
the addition formula _ ^^^y 

and carry out interpolation by a multiplication or succession of multi¬ 
plications. If, for example, is tabulated at intervals of 0-01 in x, an auxi¬ 
liary table for x — 0(0-0001)0-01 would enable values to be obtained for 
four-decimal values of x in the range of the main table by a single multi¬ 
plication; alternatively a set of auxihary tables for x = [0(0-l)l] x 10~” 
for n from 2 to 5, say, would enable values of for six-decimal values 
of X to be obtained with not more than four multiplications. 

If the function y to be interpolated satisfies a simple differential equa¬ 
tion such as y" = xy, formulae for successive derivatives of y, obtained 
by successive differentiation of the differential equation, may be simple 
enough to be used for the numerical evaluation of these derivatives. 
Then Taylor’s series can be used for interpolation between tabular 
values, f{xQ-\-x> hx) being calculated from as many terms of the series 

8x) = /o+i)(Sa:)/'o-f^+ - (5-29) 

as are appreciable. A good check is provided by evaluation of the 
alternative expansion, in terms of/and its derivatives at a: = namely: 

(where q = 1—i?) 

= + -i?=(5a:)='/r+- • (6-30) 

The convenient quantities to tabulate for interpolation purposes are 
not the derivatives but the quantities [{8x)^/k[]f^^\ sometimes called 
‘reduced derivatives’. This method of interpolation is particularly 
convenient in the case of functions which have been evaluated by 
integration of the appropriate differential equation by the Taylor series 

f Chambers'a 6-Figure Tables, vol. 2 (1949), Introduction, p. xxx. 
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method (§ 7.4), since the reduced derivatives are evaluated in the course 
of this calculation, and so are available for interpolation purposes with¬ 
out any further work. An example is provided by the tablesf of the 
function usually written Bi(a;), which is one solution of the equation 
y" = xy. 

5.6. Subtabulation 

Subtabulation is a special case of interpolation, of which the purpose 
is to take a function/(a;) at tabular interval 8 x and construct from it a 
table at a smaller tabular interval sSz; in practice s is usually or 
The values of the function at the large interval {Sx), between which 
interpolation is carried out, are called ‘pivotal values’ of/(a;). 

It is a valuable process when the direct calculation of f{x) is difficult 
or long, for example by summing a series of a large number of terms, 
or evaluation of a definite integral in which the integrand is a function 
of X. In such cases we want to restrict the number of values of x for 
which f{x) is calculated directly, and derive from them a table at a 
smaller interval, probably one in which second differences at most have 
to be included in interpolation, and possibly one in which linear inter¬ 
polation is adequate. There is no purpose in subtabulation if linear 
interpolation is already adequate; the purpose of sub tabulation is to 
break down the tabular interval when linear interpolation is certainly 
not adequate; so it is essentially concerned with non-linear interpolation. 

Since 8”/ is of order {8x)^, the higher-order differences are very much 
reduced by even a moderate degree of subtabulation. For example, 
subtabulation to fifths reduces fourth differences by a factor of over 500 
and sixth differences by a factor of over 15,000. 

In subtabulation, a systematic set of results is required, instead of 
an isolated result as is more usual in an interpolation process. This 
suggests that a systematic procedure should be used for obtaining the 
results. 

We could interpolate a sequence of values of the function itself and 
check the differences; alternatively we could construct the sequence of 
second (or higher) order differences of the function at the smaller interval 
and build up from these, using the facilities for building up a function 
from its differences (the ‘National’ machine, for example, if available). 
The latter process has the advantages that (i) most of the work is done 
with small numbers, and (ii) a good overall check is provided by the 
reproduction of the pivotal values in the course of the summation of the 

t British Association Mathematical TableSy Part-volume B, The Airy Integral (1946). 


78 


INTERPOLATION 


differences of the function at the smaller interval. The former does not 
check that the correct pivotal values have been taken, and the use of 
incorrect pivotal values may not be apparent in the differences of the 
subtabulated function; these differences check against random errors, 
but the effect of an incorrect pivotal value is a systematic error, and 
may not be indicated by differences. This is illustrated by the following 
example. 

Consider the subtabulation to tenths of sin x from a table at intervals 


of 10°. 

Use of am 

incorrect value at a: = 50° 

(0-76624 for 0-76604) might 

give a 

set of subtabulated values as 

follows 

: 



X 

‘sino;’ 



X 

‘sin a:’ 



30° 

0-50000 

] 504 


50° 

0-76624 

1110 

— 23 

31 

1504 

1488 

-16 

51 

7734 

1086 

-24 

32 

2992 

1472 

-16 

52 

8820 

1061 

-25 

33 

4464 

1455 

-17 

53 

0-79881 

1036 

— 25 

34 

5919 

1438 

-17 

54 

0-80917 

1010 

-26 

35 

7357 

1420 

-18 

55 

1927 

986 

-24 

36 

0-58777 

1402 

-18 

56 

2913 

961 

-25 

37 

0-60179 

1385 

-17 

57 

3874 

935 

-26 

38 

1564 

1367 

-18 

58 

4809 

910 

-25 

39 

2931 

1348 

-19 

59 

5719 

884 

-26 

40 

0-64279 

1329 

-19 

60 

0-86603 

858 

-26 

41 

5608 

1309 

-20 

61 

7461 

832 

-26 

42 

6917 

1289 

-20 

62 

8293 

806 

-26 

43 

8206 

1269 

-20 

63 

9099 

780 

-26 

44 

0-69475 

1248 

-21 

64 

0-89879 

752 

-28 

45 

0-70723 

1226 

-22 

65 

0-90631 

724 

-28 

46 

1949 

1203 

-23 

66 

1355 

695 

-29 

47 

3152 

1181 

-22 

67 

2050 

668 

-27 

48 

4333 

1158 

-23 

68 

2718 

640 

-28 

49 

5491 

1J33 

-25 

69 

3358 

611 

-29 

50 

0-76624 


-23 

70 

0-93969 




Here the pivotal values are underlined; that at a: = 50° is in error by 
20 units in the fifth decimal, the others are correct to five decimals. 
The differences of the subtabulated values are no more irregular than 
would be expected as the result of rounding errors, and certainly contain 
no suggestion of an error of 20 units. 

Certainly such an error in a pivotal value ought to be detected by 
differencing the pivotal values before beginning the subtabulation. But 
if a Lagrangian formula were used for carrying oxit the subtabulaticn, 
this step might be omitted since the differences of the pivotal values 
would not be used in the subtabulation process; if they were to be 
obtained to check the pivotal values, it would be better to use them in 
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the subtabulation also. The point of this example, however, is to show 
that differencing the subtabulated values does not by itself provide an 
adequate check of the subtabulation process. 

5.61. End-figure method of subtabulation 

Comrief has given a convenient process for subtabulation, in which 
only the last digit of each interpolated value is evaluated by the use of a 
suitable interpolation formula, and the complete values are then built 
up from their differences. From the last digits in the interpolated func¬ 
tion values, only the last digits in the differences can be obtained directly; 
but in subtabulation at a fraction s of the interval between the pivotal 
values, the nth differences of the subtabulated values are approximately 
s" times those of the pivotal values, and for some value of n these wth 
differences of the subtabulated values will vary slowly enough for their 
last digits to establish the values of the differences themselves. Suppose 
for example that, for the pivotal values, =610 and Sy+i = 505, 
and subtabulation is to fifths (s = g). Then the second differences of 
the subtabulated values will be approximately jg of those of the pivotal 
values, that is (allowing for possible effects of rounding errors) from about 
25 at the beginning of this interval to 19 at the end. Hence if the last 
digits of the second differences of the interpolated values are 

5, 3, 3, 1, 0, 

these second differences can with confidence be given the values 

25, 23, 23, 21, 20, 

and the function can then be evaluated by summation from these values 
(see § 4.46). If any mistake is made, it is shown up by the pivotal values 
not being reproduced in the summation. 

The process can be illustrated in a simple case by example (6) of § 5.43 
(see p. 73). If in that example the values of /(0-665) and /(0-666) had 
been evaluated in full, but only the last digits of /(0-667) and /(0-668) 
had been determined, the results (rounded off to five decimals) would 
have been: 

X 0-665 0-666 0-667 0-668 

f{x) 1-50376 1-50150 .5 .0 

Differences at the 
smaller interval 

t L. J. Comrie, Monthly Notices, R.A,S., 88 (1928), 506; Interpolation and Allied 
Tables, incorporated in Nautical Almanac, 1931. 
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where the dots represent digits so far undetermined. But, as mentioned 
on p. 74, the second differences of / at the smaller interval are about Og 
so each of the first differences whose last digit is 5 must be —225. The 
values of f{x) at a: = 0-667 and 0-668 could then be built up from these 
differences. 

The following is a more extensive example, in which it is necessary to 
go to second differences before writing down the values of a set of differ¬ 
ences from their last digits, and which also illustrates some further 
points of procedure. 

Example: Given the following table: 


X 

/(^) 




0-310 

2-96671 

8476 



0-305 

3-05147 

8816 

340 

20 

0-300 

3-13963 

9176 

360 

21 

0-295 

3-23139 

9557 

381 

22 

0-290 

3-32696 

9960 

403 

26 

0-285 

3-42656 

10389 

429 


0-280 

3-53045 





to subtabulate to fifths (i.e. at intervals 0*001 in x) from x = 0*300 to x ~ 0*290. 

(Note: for this function it is convenient to take differences in the direction of 
X decreasing, as in this table; then, apart from effects of roimding errors, they are 
all positive.) 

Everett’s formula to second differences (unmodified) is adequate in this case; 
the Everett coefficients for the points of sub tabulation are: 

r) 0 i A At 

P ^ 6 ft 6 6 A 

E^ip) 0 -*048 -*064 -*056 -*032 0 

^2{P) 6 -*032 -*056 -*064 -*048 0 

The second differences of the pivotal values over the range specified for the sub- 
tabulation are from 360 to 403, so for subtabulation to fifths the second differ¬ 
ences of the sub tabulated values will be from about 14 to 16. Thus it is only 
necessary to difference the end digits to second differences in order to be sure of 
the values to be used in building up the function values. Since this building up is 
to be done from second differences, we will need two function values from which 
to start, namely the pivotal value/(0*300) and the value/(0*299). For the latter, 
Everett’s formula gives: 

/(0-300) 3*13963 

+ i8/(0*2995) 18352 

+ E^a)8j{0-300) -173 

+ F,a)8^f {0-295) -123 

/(0-299) = 3 - 157687 , or, rounded off, 3-15769. 
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Only the rounded value is needed subsequently; guarding figures are kept in the 
contributions to it, but can be discarded in the interpolated value itself. Guarding 
figures, written as suffixes, will similarly be kept in the contributions to the other 
subtabulated values. 

For the interval x = 0-300 to 0-295, the last two digits of 8 / for the pivotal values 
are 76, so the first differences of the linear contributions fo+pS/i to the interpolated 
values in this interval end with Sg. The end figure of/(0-300) is 3, and successive 
additions of 5.2 give for the end figures of/(0-300) to /(0*295) inclusive the values 

^0 

of which the last is the end figure of the pivotal value /(0-295); this provides a 
check on the additions. For the next interval the first difference of the pivotal 
values ends with 67, so the first differences of the linear contributions to the 
interpolated values end with I 4 , and for/(0-295) to/(0‘290) inclusive are 

^69 

the comparison of the last of these with the pivotal value/(0-290) again furnishes 
a check. 

The complete calculation can be arranged in tabular form as follows (S is used 
for the central-difference operator at the smaller intervals): 

Last figure 













sy 

sy 

S/ 


X 

8^0 

SYi 


(a) 

ib) 

(c) 

id) 

/ 

S/ 

/ 

0-300N 




^0 




^0 

3 

6 



1806 

3-13963 

299 




82 

-17, 

-123 

= 

87 

9 

0 

4 

14 

1820 

6769 

298 

» 360 

381 

62 

34 

-23o 

- 2 I 3 


9i 

9 

6 

5 

16 

1836 

7589 

297 




8 , 

Cl 

0 

1 

-24, 

= 

^0 

4 

0 

5 

16 

1850 

19424 

296 




38 

-Hs 

-I 83 

=1 

^0 

4 

5 

5 

16 

1865 

3-21274 

296 j 




9o 



= 

9o 

9 


6 

16 


3139 











0 



1880 


294 \ 




O 4 

-183 

-13o 

= 

9i 

9 

6 

6 

16 

1896 

6019 

293 




Is 

-24, 

- 22 , 

= 

4, 

5 

1 

5 

16 

1911 

6916 

292 

> 381 

403 

u 

^2 

- 2 I 3 

-26o 


5* 

6 

7 

6 

16 

1927 

28826 

291 




^0 

-123 

-196 

= 


3 

3 

6 

16 

1943 

3-30763 

0-290 J 




60 




60 

6 





3-32696 


(а) Linear contribution/o+joS/j, end figure and guarding figure only. 

( б ) ,(c): and 

(d) Sum of (a), ( 6 ), (c), end figure and guarding figure only. 

The column headed ^ 8 /gives the last figure and guarding figure, as required for 
building up the linear contributions to the interpolated values in column (a). 
Columns ( 6 ) and (c) give the second-difference contributions to the interpolated 
values; these are here given in full as this makes the work easier to follow; however 
only the last full digit and a guarding figure are necessary. The sums of entries 

o 


6013 
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(a), (6), (c) with guarding figures are given in column (d) to illustrate the procedure, 
but only the roimded values of the last digit, as given in the next column, are 
required for the subsequent work. These rounded values are then differenced to 
second differences, and from the result we have already had, that the values of 
are about 14 to 16, the complete values can now be written down from their last 
digits. Then from/{0*300),/{0*299) and these second differences, the values of/ 
can be built up. A thorough check is provided by the reproduction of all the 
pivotal values. 

Notes: (i) In this example,/(a:) is the function [{l/a?)>'—l]/y with y — 1*4. The 
pivotal values were calculated from this defining formula; for the intermediate 
values subtabulation is a mueh quicker and easier process than evaluation of this 
formula. 

(ii) It is not necessary to carry out the full evaluation of/{0-299) by interpola¬ 
tion. The value of S/(0*2995) is approximately i^[/(0-305)—/(0-295)] which is 
1799—rather larger since 8^ is positive—and its last digit is 6, which indicates the 
value 1806 with some certainty. But even if a wrong value were taken, this would 
be shown by the pivotal value/{0*295) not being reproduced; and from the amount 
of the discrepancy, the corrections to be made can easily be determined. For 
example, if 8/(0-2995) were taken as 1796 instead of 1806, each of the first five first 
differences would be in error by —10, so the discrepancy between the value of 
/{0*295) obtained by building,up and the pivotal values would be —50; this would 
indicate that each of these first differences must be increased by 10. (This correc¬ 
tion process is not available if the subtabulated function values are built up from 
differences of higher order than the second.) 

(iii) If the second differences of the subtabulated values vary too rapidly for their 
last figures to be a certain indication of their complete values, a corresponding 
process involving building up from third or fourth differences can be used. Alter¬ 
natively, the process could be carried out with the last two digits in each sub- 
tabulated value instead of with the last digit only. 

Another method of subtabulation, also suggested by Comrie,t involves 
the direct calculation of second or fourth differences of the subtabulated 
values from formulae relating differences at interval s Sx to differences 
at interval Sx. The subtabulated values are then built up from their 
differences. 

5.7. Interpolation of a function given at unequal intervals of the 
argument 

For the interpolation of a function given at unequal intervals of the 
argument, the interpolation formula usually used is that of Lagrange. 
This is based on the use of an nth degree polynomial which takes the 
given function values at (n +1) values of x. Such a formula is called an 
(n-j-l)-point formula. An even number of points (odd value of n) gives 
an equal number on either side of the value of x for which the inter¬ 
polation is to be carried out. 

t L, J. Comrie, Journ. Roy. Slat. Soc.y Supplement 3 (1936), 87. 
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If the function values Are/o,/i,..,,/„ at a; = Xq, x^,...,Xh, not necessarily 
equally spaced, the polynomial of lowest degree which takes these 
values is 

\*^0 *^0 *^2 *^0 \^1 ^0 ^nf 

this polynomial is of course not the function / itself, unless / is a poly¬ 
nomial of degree n or lower; it is the polynomial which coincides with / 
at X = Xq, Xj,,.., a:,^. Interpolation is done by evaluating this polynomial 
for the intermediate value of x. 

If/is a polynomial of degree n or less, then F = / and the interpolation 
formula (5.31) is closely related to the expansion in partial fractions 
oi fj{x—XQ){x—Xi)...{x—x^). For if g{x) = 0 has roots x — a;^, x-^,..., a;„, 
all distinct, the expansion oif{x)jg{x) in partial fractions is 

■/M=: y /(^j) 

and this, applied to g{x) = (x—X(f)(x—Xj^)...{x—x,J, is 

_ f ___ fo _ L.+... 

(x—Xo){x—Xi)...(x—x,^) {Xo—Xi){Xo—X 2 )...{Xf^—xJ x—x^'^'" 

which is just another form of formula (5.31). 

Lagrange’s interpolation formula (5.31) is not restricted to functions 
given at unequal intervals of the argument. Its application when the 
intervals of x are equal has already been mentioned in §§ 5.5 and 5.6, 
and its disadvantages in that context pointed out. For functions not 
tabulated at equal intervals, however, some form of it may be the only 
method available. It is then important to systematize the work of 
evaluating the polynomial, since if it is not done in a systematic way 
it is easy to make a mistake, and adequate checking is at best difficult. 
One scheme of working, in which the coefficients of the various values 
of fj in formula (5.31) are first calculated and checked and are then used 
to form the sum (5,31), has been given by Comrie;t an important feature 
is the check of the coefficients which is provided. In another type of 
method, suggested by Aitken,J the r^^sult is obtained by a sequence of 
steps each of which is similar to a linear interpolation; this is considered 
in § 5.71. Another way of arranging the work is considered in § 5.72. 

t L. J. Comrie, Chambers''s 6-Figure Tables^ vol. 2 (1949), Introduction, p, xxxi* 

X A. C. Aitken, Proc, Edin. Math. Soc., ser. 2, 3 (1932), 56. 
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5.71. Evaluation of Lagrange’s interpolation formula by a 
sequence of linear cross-means 

Linear interpolation or extrapolation of/(x) from the values/„, /j,, 
of / at X = Xa, Xj, respectively, gives 

(Xfc— x)/a+(x— Xa)/ft 


fix) = 


Xb—Xa 


(5.32) 


Aitken calls this quantity the ‘linear cross-mean’ betweenand/^; 
let it be written /^^^(x). Linear interpolation between the values fa,b{x) 



and fb,c{x), regarded as values of an auxihary fimction at x = x^ and 
X = Xc respectively, gives 

ixc-x)fa.bix)+ix-xjff,jx) 


fix) = 


^c-^a 


Let this be writtenIt can be verified that this is the value of 
/(x) given by a three-term Lagrange interpolation formula using the 
values of / at X = x^, Xj,, and x^., 

In general, let fa,b,c...j,k,ii^) ^ numbers obtained by successive 

use of the linear cross-mean formula 


^ /...x _ iXk-x)faAc...ijix)-hix-x^)fb^,,.i,i,kix) 


(5.33) 

(n suffixes) '^k 

Then it can be proved by induction that fa,b,c...i.j.kix) is the value of 
/(x) given by an n-point Lagrangian interpolation formula using the 
values of/at x = x„, Xj,,..., Xp Xj^, which need not be in monotonic sequence. 

The process suggested by Aitken consists of forming the linear cross- 
means/o i(x),/o 2 (ic),/o^ 3 (x),... andingeneral/o j(x), then using these to form 

/o,i,2(^)./o,i,3(«).-andingeneral/o,i.j(x),then/oi2,3(^)./o,i,2,4(a^),-Jo,i,2.,(^) 

and so on. A graphical representation of the formation of the first 















INTERPOLATION 


85 


set of linear cross-means fojix) is shown in Fig. 6. An alternative 
order of procedure! consists of forming first and in 

generalthen/Q j 2 (tr),/j 2 3 (tr),... ,fj,j+ij+ 2 {^) and so on. The forma¬ 
tion of the first set of linear cross-means fjj+i{x) in this procedure is 
shown in Fig. 7. 



In using the latter procedure, the work can be arranged as follows: 


X 


X — Xj 
X — 


A 

/o 


/;j+i(^) Jj.)+i.3+Ax) Jj.3+lj+2,j+z(^) 


/o.i. 2 .s(^) 

fl.i.zAx) 



This layout is similar to that of a difference table, though the entries 
are not differences. The entries used in forming /i, 2 , 3 , 4 (^) are enclosed in 
‘boxes’, and the way in which they are selected is shown by the lines 
joining them to the entry /i, 2 , 3 , 4 (^)- 

This process has the advantages 

(i) successive calculations are all repetitions of this simple process 
of forming linear cross-means; 

(ii) the results provide their own criterion of when the process has 
been carried far enough; 

(iii) common leading figures in two of the linear cross-means can be 
suppressed in taking higher cross-means. 

t E. H, Neville, Journ. Indian Math. Soc. 20 (1934), 87. This paper includes an 
extension of the procedure to use known values of derivatives of/(z). 
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The process can be used on functions tabulated at equal intervals of 
the argument, but takes no advantage of this uniform tabulation. 


5.72. Divided differences 

For the treatment of polynomials at unequal intervals of the argument, 
it is sometimes convenient to use what are called ‘divided differences’. 
If Xq, x-i,..., and in general Xp are the values of x at which the function is 
tabulated (not necessarily in monotonic sequence), the first-order 
divided differences are 


fi^i 


O’^ll — ’ .V. .V. ’ 


Xi—Xo 


Xo — 


Xj+l) 


fiXj-+i)—f{Xj) 


(5.34) 


The second-order divided differences are 

f{Xi,X2)—f{Xo,Xi) 


f{x^,x^,x^) = 


X^ — Xq 


fir r r i _ f{^j+i>^j+2) 

and in general, the nth-order divided differences are 

-1 -• ^O.OO; 

For a function tabulated at equal intervals (Sx) of x, the divided differ¬ 
ences become the quantities 8'^fjnl (Sx)^. The main property of divided 
differences for practical purposes is that the nth-order divided differences 
of a polynomial of the nth degree are constant. 


Another application of divided differences is in the derivation of interpolation 
formulae. Such formulae for equally spaced arguments can be derived from an 
interpolation formula in terms of divided differences which is equivalent to 
Lagrange’s interpolation formula; but this derivation does not take advantage 
of the equal spacing of the arguments until a later stage, and also does not 
exhibit the relation between the coefficients in interpolation formulae such as 
those of Bessel and Everett and the coefficients in the expansions considered in 
§ 6.3. The derivation of these formulae by the use of finite-difference operators 
seems preferable. 

To show that the Tith-order differences of a polynomial of the nth 
degree are constant, consider first some properties of divided differences. 
The relations (5.35) defining the successive orders of divided differences 
are all linear in the function values/y, hence any divided difference is a 
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linear combination of a set of function values with coefficients which are 
functions of the Xj’s only. It can easily be proved by induction that if 
j ^ k the coefficient of in is 

li+n 

1/ri' 

the dash in n' indicating that the factor with i = kis omitted from the 
product; if k is outside the range j to the coefficient of is zero. 
The value of this coefficient is unaltered by a change in the order of the 
factors, and it follows that for any function f{x) the value of a divided 
differencedepends only on the values of x^ involved, 
and not on the order in which they are taken. 

Now consider the function f{x) = x’b The first-order divided differ¬ 
ences are given by 






—1 I ^ ^n-2 I I ^n-2^ ^'rV 
_|_ 1 \ I • • • I -f 1 1 


,n — l 


a homogeneous polynomial of degree {n —1) in Xj, ccy+i- Similarly 
f{Xj;Xj_f_i,Xj_f_ 2 ) is a homogeneous polynomial of degree {n —2); and by 
induction it can be shown thatx^+j,..., a;y+„,) is a polynomial of 
degree n —m. For consider the difference 


f{^j+V‘'‘’^j+ky^j+k+l) f(^j>^j+V'>^j+k)‘ (5.36) 

Since divided differences are independent of the order in which the values 
of Xj are taken, this difference is 

fi^3+k+lf ^}+V'f ^j+k) fi^j’ ^j+V'y ^j+k)’ 


and this is zero if = Xj. Hence if the A:th order divided difference 

f{Xj,X 3 -+i,---,Xj+)c) is a polynomial in Xj,...,Xj+j^, the difference (5.36) con¬ 
tains {xj+ic+i—Xj) as a factor, and the divided difference/(a^j^a^j+i,..., 
^i+fc+i) is a polynomial of degree one lower than f{Xj,Xj+i,...,Xj^,.). 
Now for f{x) = x'^, fixj,Xj+i) is a polynomial of degree (ti— 1), so 
f{xp Xj+i, Xj.+^) is a polynomial of degree (n—2), and so on. In particular 
f{Xj;Xj_^.i,...,Xj.+J is a polynomial of degree zero, that is a constant; it 
is therefore independent of the values of Xj,...,Xj-^,i, and has the same 
value as if these were equally spaced, namely, 1. 

For a polynomial of the nth degree with leading term aQX‘^, the nth- 
order divided differences of all terms but the leading term are zero, so 
the nth-order divided differences of this polynomial are constant and of 
value Oq. 

The result that for a polynomial of degree n, the nth-order divided 
differences are constant, can be used to verify whether a set of (n+m) 
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values off can be fitted by a polynomial of the nth degree. It can also 
be used to determine values of this polynomial for other values of x, 
and so to carry out interpolation. 

The latter calculation can be done by a process of building up from 
Tith-order divided differences, rather in the way in which a polynomial of 
the nth order can be built up, at equal intervals in x, from its nth. differ¬ 
ences (§ 4.42). Further, it is possible to determine derivatives of this 
polynomial, as follows. 

If thenf{xj,Xj+i) — f'{Xj)-{-0{€), and in the limit e 0, 

f{Xj,Xj) =f'(Xj). Although/(a:,-, cannot be evaluated directly from 
the values off and the definition (5.34) of divided differences, it can be 
built up from higher orders of divided differences and so determined in 
this way. Similarly 

f{Xj,X^,Xj) = 

and in general f{xj, Xj,..., Xj) = A/(")(a;^.). 

n +1 arguments 

Example: To show that the following values of/(a:) are consistent with f(x) being 
a cubic in x, and to find/(6),/'(6),/''(6) for this cubic: 

X —1 0 2 3 7 10 

/ —11 1 1 1 141 561 

The working can be arranged as follows: 

X 

-1 

0 
2 
3 
7 

12 . 

6 

6 
6 

The working above the inclined lino is concerned with showing that the third-order 
divided differences are constant, as is necessary for a cubic; that below the line is 
concerned with the evaluation of this cubic and its derivatives at x = 6. The 
arrows indicate the sequence in which the numbers in the lower part are obtained. 
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In the first part, the divided differences of successively higher orders are calcu¬ 
lated directly from the definition; for example: 


/(-1,0) = L L = 12; f{0,2) = 


O-(-l) 

/(- 1 , 0 , 2 ) =. 


0-12 -12 


2-(-l) 


1-1 

2-0 

-4, ■ 


= 0 ; 


In the second part, the divided differences of successively lower orders are built up 
from those of higher orders. The value 6 of x for which J{x) is wanted is written 
as the next value of x in the table. The value of/(3, 7,10, 6) must be the constant 
value 1 for this cubic, that is 


/(7,10,6)-/(3,7,10) 


= 1 


6-3 

so that /(7,10, 6)-/(3, 7,10) 3. 

This value is added to /(3, 7,10) = 15 to give/(7,10, 6) = 18. Then 
fn 10 gs _ /(iQ>iQ) 6)-/(7,10) 

so that /(10,6)-/(7,10) = -18 

and this is added to /(7,10) = 140, to give/(lO, 6) =: 122. Finally 

/(6)-/(10) /(6)-/(10) 


/( 10 , 6 )=’ 


6-10 


— 4 


so that/(6)—/(lO) = —488, which added to/(lO) gives/(6) = 73. 

This value of/(6) can be checked by taking it, with the values of/(3),/(7), and 
/(lO), as given values of the cubic, and using them to obtain/(2) in a similar way; 
the value obtained should reproduce the value which was used in forming the 
divided difference tabic used in the evaluation of /(6). 

To obtain /'(6) we put a second value of 6 for x, and repeat the process as far as 
the first-order divided difference only, and for we put a: = 6 again and repeat 

the process as far as the second-order divided differences. The result gives the 
cubic in powers of (x—6); in this case 

/=/(6)+/'(6)(a:-6) + ir(6)(a:-6)Hi/'"(6)(a:-6)3 

= 73 + 54(a:—6)+l3(a:—6)2 + (a:-6)*, 

and the evaluation of this for the values of x for which the function values are 
originally given checks the whole calculation. 


5.8. Inverse interpolation 

The problem of inverse interpolation is this: given a table of f{x) as 
a function of x, to find the value of x for which f{x) has a specified value. 
If the table is not at equal intervals in x, there is no distinction between 
direct and inverse interpolation; the following applies to tables at equal 
intervals in x, as is the case in almost all tables. 

The table can be regarded as one of x at uneqvul intervals of f{x), and 
a method of interpolation of functions given at unequal intervals of 
the argument (§ 5.7) can be used for inverse interpolation. This process 
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takes no advantage of the equal intervals in x, and needs care in use; 
an example of how not to use it is given below (§ 5.81). 

Bessel’s formula to third differences, using modified second differences, 

S, =nxo+P&x) =/o+p8/, + B,(p)(SJ./o+5»/t)+-B3(P)8y,; 

in inverse interpolation,/(a:o4-pS^) i>s given and this equation is to be 
solved for p. If the third-difference contribution can be neglected, it 
is a quadratic forp, and could be solved as such, using the conventional 
formula, but this is a laborious and unsuitable method for practical work; 
it has been said that ‘nobody but a mathematician would do it that way’. 

One method is to determine p roughly by means of a graph or a few 
trial direct interpolations, and then make a small subtabulation in the 
neighbourhood of the rough value of p, at such an interval that linear 
interpolation can be used for the final step. This method may be found 
the best for occasional isolated inverse interpolations, and in the neigh¬ 
bourhood of turning values of f{x). 

Another method is to write Bessel’s formula in the form 

P = lf,.-f«-B2(pWo+Sl,A)-BApWfmi (5.37) 
and use an iterative method, improved if required by the process of 
‘exponential extrapolation’ (see§§ 3.4 (a) and 9.32). If second differences 
are modified by the use of the throwback from fourth differences, as 
indicated in formula (5.37), this can be used provided fourth differences 
are less than 500. If they are greater than this, it would be best to 
proceed by means of some preliminary subtabulation, for values of p 
in the neighbourhood of that given by (5.37). 

The accuracy to which p can be determined depends on the number 
of figures in 8/j and in assessing this accuracy it must be remembered 
that the last digit of 8/j may be affected by rounding errors to the extent 
of i 1. Thus a value of 8/j of about 200 is necessary to establish a second 
decimal in p. 


Example: Giv'en the following table, find sin~i0'4 in degrees and decimals. 


X 

f{x) — sin a: 


sy 


sy 

-0-1848y 

KJ 

0 

0 

17365 

0 

-528 

0 

0 

0 

10° 

017365 

16837 

- 528 

-511 

17 

-3 

- 531 

20° 

•34202 

15798 

-1039 

-480 

31 

-6 

—1045 

30° 

•50000 

14279 

-1519 

-435 

45 

-8 

-1527 

o 

0 

•64279 

12325 

-1954 





o 

O 

lO 

0*76604 
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For interpolation in the interval x = 20® to 30°, we have 

/o= 34202 
8 /i = 15798 
8 m/o+%/i = -2572 
Sy, = - 480 

in terms of the fifth decimal as unit. We wantp for/ = 0*4,/—= 5798 in terms 
of the fifth decimal. Hence, substituting in (5.37), 

p = [5798 + 2572Bj(p) + 480B3(p)]/15798 

= 0-3670i + 0 1628i?2(p)+0-0304B3(p). (5.38) 

A nominal fifth decimal is kept here in but not in the other terms since 

the quantities B^(p), B^(p) by which they are multiplied are less than 

The first term in (5.38) is the value of p which would be obtained by linear 
interpolation. Taking it as a first approximation to p, the iterative process is as 
follows: 

p r.h.s. of (5.38) 

0-367 0-3670i -f (0-1628)( — 0-05808) + (00304)( + 0-0051) 

= 0-3670i —0-0094g + 0-000l5 = 0-3577o 
0-3577 0-3670i + (0-1628)(-0-05744) + (0-0304)( + 0-0054) 

= 0-3670i —0-00935 + 0-0001g = 0 - 3518 ^. 

The change in the v'alue of the right-hand side of (5.38) is only about of the 
change in the value of p, so the value 0-35782 would not be changed by more than 1 
in the fifth decimal (due to rounding errors) if the right-hand side were evaluated 
for the better approximation p = 0-35782- The number of figures in 8/j is not 
enough to determine the fifth decimal in p to scv'eral units. According to the 
purpose for which the value of sin"^ 0-4 was wanted, it could be rounded off to four 
decimals, or the fifth retained as a guarding figure; if the latter course is taken it 
would be advisable to write it as a suffix, as a reminder that it is subject to an 
uncertainty of several units. Thus the result would be wTitten sin"^ 0-4 = 23-5782°. 

For the worker who is fortunate enough to have the use of two machines 
simultaneously a convenient way of carrying out this successive approxi¬ 
mation has been devised by Comrie.f 

Care is nece.ssary when carrying out inverse interpolation near a 
stationary value of the tabulated function. In such cases it is advisable 
to carry out a preliminary subtabulation so that in formula (5.37) 
greater than ^S/j, before carrying out the interpolation. 

5.81. How not to do inverse interpolation 

The following example illustrates the dangers of trying to carry out 
inverse interpolation by using a Lagrange interpolation formula for x 
in terms of f{x).X 

t See Chambers's G-Figure Tables, vol. 2 (1949), Introduction, p. xxix. 

J The warning provided by this example seems necessary, as this method has been 
recommended without qualification in a book on finite differences, and, moreover, in a 
context very similar to this example. 
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Example: Given the following table 

x=0 1 2 3 4 

y = 0 1 8 27 64 

find X for y = 20. 

The five-point Lagrangian formula for x in terms of i/ is 

^ r (y-8)(y-27)(y-64) (y-l)(y-27)(y-64) ^ 

l.(-7)(-26)(-63) ■ 8.7.(-19)(-56) ‘ 

, (j/—1 )(j/- 8)(2/-64) » , (y-l)(j /-8)(y-27) J 
+ 27.26.19.(-37) 64.63.56.37 ^ - ' 

and evaluation of this for y = 20 gives x = — 1-316, instead of the correct value 
(20)^ == 2-71442. The result is not appreciably improved if one takes a six-point 
formula by including the value a: = 5, y = 125 so that there are three points on 
each side of the value for which the interpolation is carried out; and a better result 
(a; = -f-2-923) is obtained if a three-point formula involving only the values a; = 1, 
2 , and 3, is used in preference to the five-point formula. 



The reason for this discrepancy is that x — cannot be represented 
adequately by a polynomial in y over the range in question, whereas 
Lagrange’s interpolation formula is based on a polynomial approxima¬ 
tion to the function being interpolated. Fig. 8 shows a comparison 
between x — y^ (full curve) and the quartic polynomial (5.39) by which 
the Lagrangian interpolation is carried out (broken curve). 
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If this method is used for doing inverse ihterpolation, it is advisable 
to check the resulting value of x by doing a direct interpolation in the 
table of/(.r) for that value of x, and to verify that this reproduces the 
value of f{x). 

5.9. Truncation errors in interpolation formulae 

Except for polynomials of degree not greater than n, an interpolation 
formula to rith differences, or an (7i+l)‘Point Lagrangian interpolation 
formula, is only an approximation. All the formulae for direct interpola¬ 
tion which we have considered express the interpolated value f(x) of 
a function as a linear combination of tabular valuesor, to put it 
another way, as the result of some linear operation on the function 
specified by these tabular values. A general method for finding a formal 
expression for the truncation error in such a formula has been given by 
W. E. IVIilne, and is considered in the next chapter (§ 6.7). 

The method of inverse interpolation considered in § 5.8 is not linear 
in/, and is not covered by JVIilne’s treatment. 


5.91. Whittaker’s cardinal function 

As already emphasized in § 4.1, the values /• of a function /(x) at the discrete 
values Xj- = arQ-f-ySa; of x do not define the function uniquely for intermediate 
values of x. However, experience of non-linear interpolation suggests that the 
intermediate values obtained by such an interpolation process specify a quite 
well-defined function, and it is of interest to inquire what this function is and 
just how it is related to /(x). This was examined by Whittakerf and later by 
Ferrar;j; this section summarizes some of their conclusions, without proofs; for 
proofs and fuller discussion reference should be made to the papers quoted. 

Their main conclusion is this. Of all the functions /(x) which have a given set 
of tabular values, there is one which has special properties which entitle it to be 
regarded in a sense as the ‘simplest’ function with these values, and this is the 
function whose values are determined by a central-difference interpolation formula 
regarded as an infinite series, as distinct from a truncated series which is only correct 
for a polynomial of finite degree. We shall consider a function /(x) which has no 
singularities for real values of x and which tends to 0 as |a;| -> oo; such a function 
cannot be represented over the whole range of x by any polynomial approximation. 

The function /(y) defined by 


/(y) = 1 for y = 0 

= (sm7ry)/7ry for y ^ 0 


(5,40) 


has the property that (f>{y) — 0 for y integral and non-zero. Hence the function 
4>\_{x—Xj)l(hx)]J{Xj) has the value f{Xj) at x = Xj and zero at all other tabular 
values of x, and a function having all the tabular values f{x^) of f{x) can be 


t E. T. Whittaker, Proc, Boy, Soc, Edin. 35 (1915), 181. 
t W. L. Ferrar, ibid. 45 (1925), 269 ; 46 (1926), 323 ; 47 (1927), 230, 
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constructed by adding a set of such contributions <f)[{pc —iry)/(8x)]/(a:j), one for each 
tabular value, thus: ^ ^[(x-x,.)/(8x)]/(x,.). (5-41) 

J 

This function, which was introduced by Whittaker, is called the ‘ cardinal function 
associated with/(x)—or, more precisely, with the set of tabular values/(^ry) of/(^) 
at intervals Sx, It is, by definition, the same function of x for all functions f(x) 
having the same tabular values. For values of x other than tabular values, 

(^[(x—Xj)/Sx] = [sin{7T(x—XQ—jSx)/Sx}]/[7r(x—Xj)/Sx] 

= (— l)^Tsin{77(x—a;o)/S:r}]/[7r(a:—Xy)/S:r^ 

so an alternative form for the cardinal function is 

C(x) = (Sx/7r)[sin{7T(x—Xo)/8x}]^f(Xj-)/(x — Xj‘), 

j 

From this form it is apparent that care may be necessary in summing the series 
for the cardinal function unless f(Xj) tends to zero sufficiently fast as |xy| ^ oo. 



The cardinal function has three main properties, in addition to having the same 
tabular values asf(x). First, if analysed into simple-harmonic components whose 
variation with x is given by cos[k{x—X q) then no components with periods 
less than 28a: (that is, with k > 7 t/ 8 x) occur. The point of this can be seen by 
considering/(a:) analysed into such components and examining what modification 
can be made in these components without altering the tabular values. If 
AjcCos[k{x—XQ)+^fc] is one such component, then there exists a value of n such 

—tt/Sx < k~2n7Tl8x < 7t/8x; 

if we write kQ — k — 2n7Tl8x, then replacement of cos[^(a:—a:o)+jSjt] by 

cos[/:o(a: —a:o)+^fc] 

does not change the tabular values; this is illustrated in Fig. 9, in which the light 
curve represents cos/:(a:—arg) and the heavy curve cos A:o(a:—arg); the ringed points 
indicate tabular values of x. Also |A:g| < rr/Sx, Now cos[fcg(a:—a:g)+jSjt] has a 
longer period than cos[k{x — XQ)+^j^]; that is, the former is less rapidly oscillating 
function, and indeed it is the least rapidly oscillating function by which 

cos[A:(a:-a:g)+jSfc] 

can be replaced without affecting the tabular values. Hence if such a replacement 
is made for each simple-harmonic component of/(a:), the result is a function from 
which rapid oscillations have been removed as far as is possible without affecting 
the tabular values; and since |A:g| ^ Tr/Sa:, this function has no simple-harmonic 
components with k > tt/Sx. This is the first of the three main properties of the 
cardinal function, and indeed is a property which Whittaker originally used in 
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constructing this function. The only functions which have this property are the 
cardinal function itself and functions obtained by adding to it a constant multiple 
of sin[7r(x—Xo)/Sx]. 

A second property of the cardinal function, established by Ferrar, is one of 
consistency, in the following sense. Let us take the cardinal function (5.41) asso¬ 
ciated with the set of values f{Xj) of f(x) at intervals Sx, and take another set of 
values of X at a different interval SjX (> 0), say the values x ~ xi = 

From the values at x = x^ of the function C(x) defined by (5.41), we can construct 
the cardinal function (7i(x) associated with the values C(xi) of C(x), by replacing 
f{x^) in (5.41) by (7(x^), Sx by SjX, and suffix j by suffix Z: 

(5.42) 

L 


Now C(x) defined by (5.41) is not in general the same function as/(x) whose tabular 
values occur on the right-hand side, and correspondingly it might seem that the 
function Ci(x) defined by (5.42) would not be the same as the function C(x) whose 
values at interval S^x appear on the right-hand side. However, Ferrar showed 
that provided SjX < Sx, and subject to certain conditions onf(xj) for large values 
of [x|, the function 6\(x) defined by (5.42) is identical with the function C(x) of 
which Ci(x) is the cardinal function. Thus for any function /(x) satisfying the 
conditions mentioned, the function C(x) defined by (5.41) has the property of 
reproducing itself in the operation of taking the cardinal function for any smaller 
interval. This property is related to that considered in the previous paragraph, 
for C(x) has no simple-harmonic components with k > tt/Sx, and so if S^ x < S.x, it 
has none with k > tt/SiX. 

A third property of the cardinal function, and the one most closely related to 
the theory of interpolation, is that, as already mentioned, it is the function whose 
intermediate values are determined by a central-difference formula, considered 
as an infinite series, applied to the tabular values of x. The cardinal function defined 
by (5.41) has the same tabular values as/(x) and could be used, instead of a Lagrange 
polynomial (5.31), for interpolation between these tabular values; it should be 
particularly suitable for such a purpose because of the absence of rapidly-oscillating 
simple-harmonic components. Whittaker showed (without use of any interpolation 
formula) that the expression (5.41) for C(x) in terms of the tabular values of f(x) 
could be transformed into one in terms of the differences of these tabular values; 
expressed in terms of Bessel’s interpolation coefficients, this result is: 


C(x,+pBx) = 2 + 

n = 0 

the right-hand side is an expression in terms of even-order differences at the 
beginning of the interval only, and odd-order differences, the coefficients of the 

latter being: B,^{p) += if>,„^,(p)/{2n+l)l 

(see formulae (4.11) and (5.24)). Written symmetrically in terms of the beginning 
and end of the interval, this result becomes 


C(x,+pSx) = I [B2„(p){S=‘7o+8^”/i} + -B 2 „+i(p)S=“”+yi]. (5.43) 

n = 0 

The right-hand side is the result of applying Bessel’s interpolation formula (5.23), 
regarded as an infinite series, to the tabular values of/(x) and their differences, 
and formula (5.43) states that the intermediate values obtained in this way are 
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the values of the cardinal function C{x); with the conditions on f{x) assumed by 
Whittaker, the series on the right of (5.43) always converges. 

A corresponding result for the Lagrange interpolation formula with equal 
intervals of x has been proved by Ferrar, relating the cardinal function to the 
limit, as n ^ 00 , of a symmetrical 2n-point Lagrange interpolation formula. 

For a function f{x) ~ cos{kx-]-^) with |fc|8a; < tt, the cardinal function C(x) 
is identical with f{x). This, and the property of the cardinal function expressed by 
(5.43), is one aspect of the property of the functions sinx and cosx, already 
mentioned in § 4.1 and § 5.21, that they can be interpolated accurately from the 
values at a very wide interval in x, pro vided, of course, that enough terms are used 
in the interpolation formula. 


VI 


INTEGRATION (QUADRATURE) AND 
DIFFERENTIATION 

6.1. Definite and indefinite integrals, and the integration of 
differential equations 

Thebe are two kinds of situation in which we may want to carry out 
numerical integration. One is the integration of a given function of the 
independent variable; this is sometimes called quadrature.f The other 
is the integration of a differential equation, which can be regarded as 
the evaluation of an integral in which the integrand at each value of x 
depends on the value of the integral at that value of x. This is represented 
formally by writing the solution of the equation dyjdx = f{x, y) as 

y = \f{x,y)dx. (6.1) 

From the point of view of carrying out the integration by numerical 
(or mechanical) means, the only difference between quadrature and the 
integration of an ordinary differential equation is that in the former 
case the integrand in (6.1) is independent of y, and so is known as a 
function of the variable of integration over the whole range of x before 
the integration is started, whereas in the latter case the integrand at 
any value of x is not known until the integration has been taken as far 
as that value of x. The present chapter is concerned with the integration 
and differentiation of given functions of x. The integration of differential 
equations is considered in Chapter VII. 

In integration of a given function of x, the results required may be of 

b 

two kinds, a definite integral J f{x) dx between a single specified pair of 

a 

X 

limits a, h, or an indefinite integral J /(^) d$ as a function of its upper 

a 

limit. The latter is much the more important, and will be considered 
first. Usually when results of this kind are wanted, they are wanted at 
the same values of x as those at which f{x) is tabulated, though occasion¬ 
ally results at twice this interval will be adequate. 

f It is also sometimes called ‘mechanical quadrature’; but this term is misleading 
since there is nothing more mechanical about the process than there is about any other 
numerical calculation. 
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6.2. Integration formula in terms of integrand and its differences 

The relation between the first differences of a function and its first 
derivative and the differences of this derivative has already been ob¬ 
tained in § 4 . 74 , where it has been pointed out that such a relation is 
an integration formula. We have 


/i—/o ~ j f 


xo 


substituting this in (4.58) and replacing/' by/we have 

Xi 

//■ 


Xo 


'dx = |(Sa:)[/o-l-/i—^( 8 ^ 0 +^!/!)+ 

+7¥o(8yo+SVi)--6'o!lo(Syo+8y^ (6-2) 

This could be obtained directly by the use of finite-difference and 
differential operators, without reference to § 4.74, as follows. Expressed 

Xi 

in terms of operators, J / da: is {E — 1 )Z)~’/q, and we want to express this 

Xo 

in the form ■|( 8 a:)</>( 8 )(^/-l-l)/Q. Hence 

|(§a:)/(8)(^+l) = (^-l)H-\ 

E—1 2 tanhiH 


or 


m = 


iu 


E-]-l U 

and the algebraical work of expanding /(S) in powers of 8 then proceeds 
as in § 4.74. 

An alternative derivation is by integration of Bessel’s or Everett’s interpolation 
formula with respeet to p, for 

Xl 1 

If dx = (Sx) \ f(Xo+p8x)dp. (6.3) 

lo p = o 

In Bessel’s formula, the coefficients of the odd order differences are odd functions 
of (p — ^), so they give zero on integration. The integrals of the coefficients of the 
even-order differences in Bessel’s formula give the coefficients in (6.2). 

The ratio of the coefficients of 8^ in (6.2) to that of S^/is — = —0*1833, which 
is very close to the value —0*184 used in modifying second differences in inter¬ 
polation by means of the throwback; this is not surprising in view of the close 
relation just mentioned between the interpolation and integration formulae. How¬ 
ever, unless modified second differences have to be calculated anyway for inter¬ 
polation purposes, use of them in the integration formula is no simpler than 
calculating the fourth-difference contribution as it stands. 

In using the integration formula ( 6 . 2 ) it is advisable to add the con¬ 
tributions in the square bracket first and finally multiply the whole by 
one-half rather than dividing each separate contribution by two 
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before adding; this halves the possible rounding error without requiring 
that any additional figures should be kept. 

For reference later, an alternative form of formula (6.2) should be 
noted. From the relation (4.34) between the operators p. and 8 it follows 
that (F-hl)8^”' = 2{E —l)p,8^”’“^. So each pair of terms (S^”/Q-i-8^”/j) 
can be written and formula (6.2) can be written 

alternatively 

Xt 

Xo 

(6.4) 

This is not as convenient as (6.2) for integration through a single interval, 
but may be more convenient for integration over a number of intervals. 


6.21. An alternative derivation 

Formula (6.2) and other integration formulae can be obtained by a 
rather different approach as follows. The simplest integration formula, 
often known as the Trapezium rule’ or the ‘trapezoidal formula’, is 


J/dx = i(S*)(/,+/,). 


Xo 


For a more accurate formula, let us write 


J/dx = i(8a:)[/o+A+Oj; 


(6.5) 


( 6 . 6 ) 


Xo 


|(8a;)(7i can be regarded as a correction to the result obtained by the 
trapezium rule. 

Now J fdx = {E—1)D-^/q and (/o+/i) = (F+l)/o, 

Xo 

so that Cj is given in terms of operators by 

Ci= («-7) 

Different integration formulae are given by different ways of expressing 

Oi. 

If we want to express in terms of the sum of contributions from the 
beginning and end of the interval, we write it as the result of an operation 
on (^+l)/o, thus: 

{E+l)fo = ( 6 . 8 ) 




2{E-i) ; 

U{E-^\) 
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and expansion of the operator here in powers of 8 gives formula (6.2). 
It is also possible to expand it in powers of U and so obtain an integration 
formula in terms of higher derivatives of /. 

If we want to express Ci in terms of the difference between contribu¬ 
tions from the beginning and end of tlie interval, we write it as a result 
of an operation on One way of doing this is to use in formula 

(6.8) the relation (4.34), namely {E-\-l)8 = 2{E—l)[x. This gives 


2 


tanhJlC/ 


— lj(^¥i—/^S/o); 


and expansion of (tanh^C/)/^C/ in powers of 8 then gives (6.4). 


6.22. Integration formula in terms of the integrand and the 

differences of its derivative 

Another way of writing Ci as a difference between contributions from 
the beginning and end of the interval is 

(6.9) 

This form for in terms of the derivative/' of the integrand and its 
differences, is convenient as the operator here is an even function of U 
and so of 8. 

From formula (4.34), {E-{-l)/{E—l) = 2ixJ8, so formula (6.9) for 
can be written 

C'l = -(2/82)(8a:)[(;a8/C7)-(8Vt/^)](/;-/'o). (6.10) 

The expansions for ifiS/U) and {8^/U^) in terms of 8 are given by 
(4.52) and (4.47) respectively; substitution in (6.10) gives 

^1 ~ 50^^+560^^ soHoO^^K/i 

so that 

Xt 

j Ax) dx = i{8x)[(/<,+/,)-j{Sx){{/i{sy;-sy;)+ 

^0 

+5lo(Syi-8y;)-3oifoo(Sri-S%)}] + 0(8a:)io. (6.11) 

An advantage of this formula is the small coefficient (|^.Jg = gig) of 
the term of order (8a;)‘i in the square bracket, compared with that {^q) 
of the corresponding term in (6.2). Further, the term of order {8x)^ in 
the square bracket only involves values of quantities at the beginning 
and end of the interval of integration, whereas the corresponding term 
in (6.2) involves the values offf and/_i outside that interval. We shall 
see later that both these advantages are important in the integration 
of differential equations. 


^■ = 5 




U E 


8x 

V 
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6.23. Integration formula in terms of integrand and its deriva¬ 
tives (Euler-Maclaurin formula) 

Expansion of the operator in (6.9) in terms of U instead of in terms 
of 8 will give an integration formula in terms of the integrand and its 
derivatives. This is known as the Euler-Maclaurin formula. 

In terms of U, (6.9) becomes 

= -{2{8x)jU%iUeothiU-l]if[-f,). (6.12) 

Now the expansion of cot in powers of 2 isf 

izcotiz= 1-1 b,z^-1b,z‘-Ib,z<‘-..., (6.13) 

the coefficients being the Bernoulli numbers; the values of the first 
few are 

-^1 — h ^2 — -®3 = — -®5 — sV 

The corresponding expansion of coth \y is given by putting z = iy 
in (6.13), so the required expansion of (2/t72)[^C7coth^C7— 1 ] is 

(2/ (7 coth = 

= i[l-A£^H2Ai|f7«-OTk|P'']+0(S*)'. (6.14) 

Substitution in (6.12) then gives an expression for Cj, and substitution 
of this in (6.6) gives the Euler-Maclaurin integration formula 

I fix) dx = i(Sx)[/„+/,-i(8x){(/;-/;)-^(8x)*(/r-/j')+ 

+i(S*)‘(/r-/5)-iiifc5(8^)*(/5'‘'-/?')}]+0(S^)®- ( 6 . 1 S) 

This formula is of limited practical value, since values of the higher 
derivatives of the integrand will not generally be available. They may, 
however, be available in two cases; first, when the integrand is given 
by a sufficiently simple analytical formula, and secondly, when /satisfies 
a sufficiently simple differential equation. The analytical formula or 
differential equation must be such that it can be differentiated several 
times w'ithout leading to expressions too complicated for practical 
numerical evaluation. 

6.3. Integration over more than one interval 

If it is adequate to obtain the values of the integral at intervals 
greater than those at which the integrand is given, other integration 

t E. T. Whittaker and G. N. Watson, Modern Analysis (C.U.P. 1927), § 7.2. The 
notation for the Bernoulli numbers used here follows that of Whittaker and Watson. 
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formulae are available. For integration over 2k successive intervals 
{hx) we have 

Xk 

J f{x) dx = = 2(sinhfcC7)Z)-yo. 

X-k 

A first approximation is 2 t( 8 a;)/o, so we try to obtain a (^( 8 ) such that 

Xk 

J fix) dx ^ 2k{8x)<f>{8)ff^. 

x^k 

The operator 0(8) required is therefore given by 

2k{8x)(f){8) = 2{sinh.k.U)lD, 

or 0(8) = {sm}ikU)jkU. (6.16) 

In particular, for I' = 1 (integration over two intervals), 

0 ( 8 ) = (sinh U)/U. 

The expansion of this operator in powers of 8 has already been considered 
in § 4.75; substitution from formula (4.52) gives 

Xl 

j f(x) dx = 2 ( 8 *)[/.+J 8 y<,-r«i 8 V«+n‘i 28 ''/o] + 0(8*)». (6.17) 

X-i 

The first two terms give the finite-difiPerence form of the integration 
formula usually called ‘Simpson’s rule’. This can be seen by expressing 
8^0 terms of function values; then the first two terms in (6.17) give 

J/(x) dx = 2(8*)[/<,+i(/,-2/„+/.,)] = J(8*)(/,+4/„+/-,]. (6.18) 

the usual form of Simpson’s rule. 

Another important formula of this kind is related to the result of 
putting = 3 in (6.16). For k = 8, 

0(8) = (sinh3C/)/3C7 = l-^^U^+?^U^-\-^,W-^0i8x)^ 

= [l + |S^+i^SH^ 8 «] + 0 ( 8 a:) 8 . 

If now we replace the coefficient by = A we obtain a formula 
which is certainly not correct to sixth differences of the integrand but 
in which almost the whole of the contribution from the sixth difference 
is included, namely 

X, 

I fix) dx = ^(8rr)[20/o+308yo+ll8yo+8Vo]-iio(8^)8«/o+0(8a:)9, 
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or, in terms of function values, taking only the terms in the square 
brackets, 

Xt 

J/(^) = to(^^)[/3+5/2+/i+6/o-|-/_i+5/_2+/-3]- (6.19) 

X-t 

This is known as ‘Weddle’s rule’. 

Another procedure for evaluating an integral over a number of equal 
intervals is to express it as the sum of a number of trapezium-rule con¬ 
tributions and a correction. 

Xn 

The approximation to J / (x) dx as the sum of a number of trapezium- 

*0 

rule contributions over intervals hx is 


(/o+ 2 /i-|-2/2- 1 -...-|- 2 /„_j+/71) 

= l{hx)[\+E)[l+E+E^+...+Er^-^)U 
= ^{hx)[{E+l){Er^-l)l{E-l)^U 

The integral itself is (-S'”—let us write it as the sum of the 
trapezium-rule contributions, plus a correction ^{8x)Gj^; that is, 

{E--l)D-Yo = J fix) dx = 

Xo 

Then 0. = (2 ^ 

SO that is related to (/„—/o) in just the same way as G^ is to (/i—/o) 
(see § 6.21). Thus we can write down three integration formulae directly 
from the results of §§ 6.2 to 6.23. 

In terms of the integrand / and its differences: 

Xn 

j f(x) dx = (8x)[J(/„+2/,+2A+...+2/„_,+/„)-A(f8/„-f.8/<,)+ 

(6.20) 

In terms of the integrand, its first derivative, and the differences of this 
first derivative: 




Xo 


dx = K 8 ^)[/o"(" 2 /i-}- 2 / 2 -t-...+ 2 /j^_,-f-/n 


8Vi) 


...)]. ( 6 . 21 ) 
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In terms of the integrand and its derivatives (Euler-Maclaurin formula): 

J J(x) dx = i(Sx)[/„+2/,+2/j+...+2/„_,+/,- 

The last is of limited practical use as a formula for numerical integration 
for reasons already mentioned in § 6.23. It is, however, useful for 
numerical work in another context (see § 11.12). 

6.4. Evaluation of an integral as a function of its upper limit 

The evaluation of an integral as a function of the upper limit can be 
carried out by successive addition of the contributions from a sequence 
of intervals {hx) covering the relevant range of x. Let us for brevity 

X 

write F{x) for | /(^) d$. Then if, for example, the integration formula 

a 

(6.2) is used, these contributions are 

^^•+1 = J /(^) == ( 6 - 23 ) 

where = (/,+/,«)-A(Sy,+S%«)+a(8V,+S%„)-...; (6.24) 

Xf\ 

and j f{x) dx is the sum of n such contributions. The summation of these 

contributions can be expressed by use of the central-sum operator a, 
the inverse of 8 (see § 4.6); operating on both sides of (6.23) with a we 

have J’-i?„=i(S*)[(w),-(«)„]. (6.25) 

In summing contributions of the form (6.23), there will be an accumula¬ 
tion of rounding errors from the corrections to the trapezium rule for the 
successive intervals. This accumulation can be made unimportant by 
the use of a guarding figure in the contributions (6.23). 

In whatever way the calculation of the integral is done, it must be 
checked. The details of the checking procedure depend on the method 
used to evaluate the integral; the following procedure, given as an 
example, refers to a calculation carried out by evaluating the successive 
contributions (6.23) and adding them. It is then advisable to carry out 
one check on the values of 5^+^ and another on the evaluation of the 
integral from them; the intermediate check of the values of will 
avoid the possibility of a large number of values of the integral having 
to be corrected if one of the values of 5^+^ is in error. 
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Operating on both sides of (6.24) with have 


(6.26) 

Values of 8^5 can be calculated (i) from successive values of 5 by the 
method of § 4.45, and (ii) from formula (6.26); comparison of these values 
provides a good check on the 5 values, and a clear indication of the location 
of mistakes, if any. When this check has been made and mistakes, if any, 
corrected, and only then, the values of should be summed, and the 
values of the integral F calculated from (6.25); the multiplication by 
should follow the summation, otherwise it may be necessary to keep 
an extra figure to avoid rounding errors. 

This summation and multiplication by ^Sx can be checked as follows. 


From (6.23) we have 




(6.27) 


and values of 8^5 are already available as they have been used in the 
process of checking the values of s. The values of the integral F can be 
checked by comparing the values of 8^F obtained (i) as S{8^F) from the 
values of F, the second differences being evaluated by the method of 
§ 4.45 and then differenced once more, and (ii) from formula (6.27). 

Various alternative checking procedures of this kind can be devised, 
for example by taking the fourth differences of both sides of (6.24) to 
give a check of the values of s. Another kind of check is an overall 
check by one of the methods considered in §§ 6.5, 6.6; but this is less 
useful, since although it will indicate the presence of a mistake it will 
not usually locate it. 


Example: To evaluate J c®* dx to five decimals at intervals 0* 1 in x. 

0 

The first seven intervals of the calculation are given on p. 107, in which each 
column, rather than each row, refers to a single value of x. The values of s are 
calculated from formulae (6.24), the contributions from the sixth differences being 
just appreciable; the second differences of these values of s are in the line below 
the values of s themselves, and the sums of the right-hand side of formula (6.26) 
occur four lines lower down. The discrepancies (60, — 121, + 61) between these 
two sets of values at x = 0-35, 0-45, and 0-55 clearly indicate a mistake, probably 
of 60 units in the last figure, at x = 0-45, and this is easily traced to the value of 
for the interval 0-4 to 0-5, which should read — 583o, so that 
s(0-46) = 2-45180. 

The differences affected by this correction should be recalculated and the 
comparison with formula (6.26) verified to make sure that the correction itself has 
been rightly made. In actual working the corrected values could be written in 
place of the erroneous values, but in this example they have been written separately 
to display both the incorrect and correct values. The successive values of s are then 
summed, and the sums multiplied by ^Sx and rounded off. Finally, this summa¬ 
tion and multiplication is checked by comparing S(S*F) with i^(Sx) 8 *s. 
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Notes: (i) In the check comparisons between the values of calculated by the 
two ways, and in the similar comparisons for 8^i^, discrepancies of a unit can be 
expected as the result of accumulated rounding errors, but discrepancies in succes¬ 
sive values should usually be in opposite directions. 

(ii) The contribution of 8®/ to 8^5 may be appreciable, although its contribution 
to s itself is negligible, and similarly for other orders of differences. 

(iii) In this case / is an even function of x, hence each difference of even order 

is an even function of x, so the even-order differences at x = 0 can be calculated 
from the formula 8*"+*/(0) = 2[S*"/(8a;)-S*"/(0)]. 


To obtain the second and higher differences at x — 0*8, the values off{x) up to 
/(l-l) have been used, though to save space they are not given explicitly. 

(iv) Each value of / is subject to a rounding error of up to in the last figure, 
so each value of s is subject to a rounding error of up to 1. If these rounding errors 
were randomly distributed, the ‘probable error’ (in the technical sense of the term 
as used in the theory of errors) of the sum of N values of s would be approximately 

so that of the result would be about JiV^(Srr) in the last significant figure of the 
values of the integrand. In the present case, with N — S, this is rather less than I 
in the sixth decimal in the integral. In making estimates of this kind, it must be 
remembered that errors up to 2 or 3 times the ‘probable error’ are not unlikely. 

(v) Another method of calculating this integral, more convenient for large values 
of X, is given in the example in § 7.3. 

(vi) The checks indicated verify the calculation from the values of the integrand 
/ but they do not check these values themselves. The differences of the values of 
the integrand form a check against gross errors, but do not check the last digit with 
certainty. An overall check on the whole calculation, including the values of the 
integrand, is provided by doing an integration by one of the methods of § 6.5 using 
another set of values of the integrand, say in this case by integrating from x i= 0 
to 0*8 using ten steps of length 0*08, or by using a Gauss integration formula (see 
§6.61). 


The following is a convenient alternative procedure, if values of the 
integrand are available at an interval 8x which is half that at which the 
integral is required. T^rom formula (6.17), neglecting terms 0(Sa;)^, 

J /(X)&= 

X,j 

SO that for integration over a range Xq to x^j, 

I'm dx = 2(8*)[ 5'A,«] 

(6.28) 

the sums being over alternate values of the integrand and of its differences 
of even order. This formula, taken as far as the term in is probably 
the most convenient form of Simpson’s rule for practical work, and the 
higher-difference terms can easily be included if appreciable. 



— 0 1 0 +0 1 0-2 0-3 0-4 0-5 0-6 0-7 0-8 

1 01005 1 00000 1 01005 1 04081 1 09417 M7351 1-28403 1-43333 1-63232 1-89648 

2010 2071 2260 2598 3118 3878 4969 6517 8727 
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Example: To evaluate J e®’ dx to five decimals at intervals 0-2 in x, using values 

0 

of the integrand at intervals 8a: = O'l. 

The integration to a: = 0-8 is given on p. 109. The integrand values and their 
differences are the same as on p. 107, but now the values of x are arranged in a 
column instead of in a line. The values of the integrand and its differences which 
are to be used are enclosed in ‘boxes’; this is a convenient way of picking out these 
values and ensuring that the right ones are used. The columns headed 2/&nd 2 8*"/ 
contain current sums of the ‘boxed’ values of/ and its differences of even order, 
and are placed on the lines corresponding to the values of x for which they will 
be used in evaluating the integral by means of formula (6.28). 

The results can be checked by a formula, similar to (6.27), obtained by taking 
third differences, at the large interval, of both sides of formula (6.28), or by one 
of the methods of §§ 6.5, 6.6. 

6.41. Change of interval length in an integration 

It may sometimes be required to change the interval length (8a:) in 
the course of an integration. Where the third and higher derivatives 
of the integrand are large, it is advisable to take small intervals, not 
only in order that the corrections to the trapezium rule should not be 
too large, but in order that the behaviour of the integrand should be 
adequately defined by those of its values which are used in the integra¬ 
tion formula. It may happen that the values of the third and higher 
derivatives vary considerably over the range of integration. If in such 
a case the interval length (8a:) which is necessary when these derivatives 
are large were used over the whole range, this would involve an un¬ 
necessary amount of work in the region in which they are small. So a 
change of interval length, or several such changes, may be required in 
the course of the integration. 

When such a change is made, there should always be an overlap 
between the ranges of x for which the different sizes of interval are used. 
This is a potent check against mistakes, which are particularly likely 
to be made at points like this at which a systematic procedure is inter¬ 
rupted. It provides a check not only against random mistakes but against 
some forms of systematic mistakes as well; if, for example, the term 
(6.24) had been taken systematically with the wrong 
sign, this would be shown up by a discrepancy between the integration 
carried out with two different interval lengths. 

In practice, the convenient intervals are 1, 2, and 5 times a power 
(positive or negative) of 10; an increase of interval length from 2.10® 
to 6.10® involves some interpolation, but this is all of the simplest 
kind, namely ‘halfway’ interpolation (see § 5.21), and should give no 
trouble. 
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6.42. Integration in the neighbourhood of a singularity of the 

integrand 

A point at which / or any of its derivatives becomes infinite will be 
called a ‘singularity’ of/. 

In deriving the integration formulae of the previous sections, it has 
been assumed that the integrand f{x) is expansible in a Taylor series 
through each interval 8x. This is not the case if there is a singularity at 
any point (including end-points) of the interval, and the approximations 
used are likely to be bad in the neighbourhood of a singularity even if it 
does not lie in the interval through which the integration is being taken. 
Examples are: 

(i) J ^ dx (integrand infinite at a; = 1); 

1 

(ri) J a;* sin a: dx (second derivative of integrand infinite at a: = 0). 

0 

A singularity can often be removed by a change of independent 
variable; for example, the substitution x — coshw makes 

r p-ax r 

— - -,dx= 

J (a:2-l)i J 

1 0 

and usually such a change of independent variable will also give results 
in a more satisfactory form for interpolation. But if the results are 
required in terms of x, and in a context in which no interpolation will be 
carried out on them, it may sometimes be better to obtain them directly 
in such a form. 

A singularity can sometimes be removed by the following process. 
We subtract from the integrand / a function ff which can be integrated 
formally and has a singularity of the same kind as /, evaluate the 
analytical formula for J ff dx, and evaluate J {f—ff) dx by numerical 
integration. This may be called ‘subtracting out the singularity’. 

This is satisfactory if the singularity is a pole of order n, but otherwise 
it may not be possible to remove the singularity in this way, though it 
may be made less severe. For example, we can write 

f rfr-c- (-^+ f ^ 

J J (**- 1 ) 1 + J (**-!)> 

1 11 

The integrand in the integral on the left is infinite at a: = 1, whereas 
that in the second integral on the right is finite, though its derivative 
is infinite. 
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An alternative, and often more effective, treatment is as follows. 
Write the integral J f{x) dx = g{x)h{x), where is a chosen function for 
which dgjdx has a singularity of the same kind as that of/. This leads 
to a differential equation for h{x) for which, however, numerical integra¬ 
tion may be quite practicable. This may be called ‘dividing out the 
singularity’. 

Consider, for example, integrals of the form 


J xPf(x) dx, 

0 

where f{x) is regular at a:: := 0 and/(0) ^ 0, and p is greater than —1 
and is not an integer. For this case the appropriate function g{x) is 
x^+'^^Kp-^l), so we write 


J x^f{x) dx 
0 


1 

pVl 


xP+^h{x) 


(6.29) 


and on differentiation obtain 


= (P+l)(/—^)- (6.30) 

On differentiating k times and putting rr = 0, this gives 
M(U(0) = (p+i)[/(U(0)-A(^-)(0)], 


so that 

mc) = 

(6.31) 

In particular 

O 

li 




h'(0) = ^/'(O) 

'>"(0) = 

P + 3 


(6.32) 


These serve as starting values for a numerical integration of equation 
(6.30). 

Numerical integration of equation (6.30) is also useful as a means of 
evaluating integrals of the form J xPf{x) dx for values of the lower limit 
in the neighbourhood of zero when p is negative and \p\ > 1. 

6.43. Integration when the integrand increases ‘exponentially’ 
A similar device of writing an integral in the form 

J f{x) dx -= g{x)h{x), 

choosing a convenient function g{x), and solving numerically the result- 
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ing differential equation for h{x) can often be applied when the integrand 
increases rapidly with x, particularly when log/(a:) increases more 
rapidly than linearly in x. This process then consists of dividing out the 
singularity at infinity. One good choice of g{x) is the leading term in 
the asymptotic behaviour of J f{x) dx] this makes ^ 1 as x oo. For 

example, J e*’ dx behaves asymptotically like e^*/2x, so we may write 
0 

J e*’ dx = {e^^l2x)h{x), (6.33) 

0 


then 



(6.34) 


This, however, is clearly not convenient for small x, and in order to 
obtain an equation applicable to the whole range of x we may be content 
to divide out the main part of the singularity at infinity by writing 

J dx = 

0 


then 


d^ 

dx 


-{-2x}i = 1, 


(6.35) 


a simpler equation than (6.34), and one which there is no difficulty in 
integrating numerically from a: = 0 (see the example in § 7.3). 


6.44. Twofold integration 

By a many-fold integration of a function/(a:) is meant a result obtained 
by repeated integration with respect to the same independent variable, 
as distinct from a double integral which is obtained by one integration 
with respect to each of two independent variables. That is, a twofold 
integral of a function / is a function F such that 






It is sometimes convenient to be able to obtain such a twofold integral 
directly without going through the intermediate stage of evaluating J fdx. 
This can be done by using formula (4.54), which for this case becomes 

= (8x)»[/„+*Sy„-rf5S%+5|,'si8%]+0(8*)'«. (6.36) 

The twofold summation of the second differences to give the func¬ 
tion values F can be done either directly by the method of § 4.46 or 
by obtaining the first differences as an intermediate step and then 
summing these. 

Effects of rounding errors may build up somewhat rapidly in this 
twofold summation, and it is advisable to carry some guarding figures. 
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The twofold integration could be carried out by two single integrations, 
one from F" to F' and the other from F' to F. Suppose that in an 
integration carried out by this method, n decimals would have been 
kept in F'. Then in a twofold integration carried out by a method not 
involving the calculation of F', the number of decimals kept in SF (or 
in F if SF is not calculated) should be enough to give n decimals in 
SFISx. 

6.5. Integrals between fixed limits 

An integral between fixed limits can be evaluated by any of the for¬ 
mulae of § 6.3 or § 6.4, the difference being that the value of the integral 
is only wanted for a single value of the upper limit. 

0-8 

Example- To evaluate | dx using formula (6.21). 

0 

In this case/ '(x) = 2xe+“* is an odd function of x, hence all even-order differences 
of/'(x) at X = 0 are zero, and their contribution to formula (6.21) is zero. For the 
even-order differences at x = 0-8 we have the following values: 


X 

0-5 

•6 

f'(x) = 2xe“’ 
1-2840 
1-7200 

sy' 

1292 

sy' 

sy' 

At X = 0-8 
f' = 3-0344 

•7 

2-2852 

1840 

238 


-ifeSy' = - 43g 

•8 

3-0344 

2626 

374 

50 

+ = + O7 

•9 

1-0 

1-1 

4- 0462 

5- 4366 
7-3776 

3786 

5506 

560 


Sum = 3*0301 


The values of/(x) are given in the example in § 6.4 (p. 107); using them we have 

2 2 = 17-33644 

/(0)+/(0-8) = 2-89648 

and, from the values of /' above, 

-i(8a:)(/'-^8y'+Tio8V') = —05050 


Hence / dx = ^(0-1)20-18242 = l-00912i. 

0 

The tolerance on this result, due to the accumulation of rounding errors of the 
function values used, is a few units in the sixth decimal. 

Some other forms which are only appropriate to an integral between 
fixed limits are considered in the following sections. In particular, there 
is the possibility of using values of the integrand f{x) not spaced at 
equal intervals in x if there is any advantage in doing so (§ 6.6). If 
the integrand is specified by a table at equal intervals in x, then an 
integration formula which makes use of this feature is usually the more 
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convenient; the interpolation necessary to give its values for use in an 
integration formula using unequa-lly spaced values of x would usually 
outweigh the advantages of such a formula. But if it is specified by a 
formula which can equally well be evaluated for any value of x, then 
use of values of x at unequal intervals may become practicable. 

6.51. Gregory’s formula 

The integration formula (6.20) expresses the correction (7„ to the sum of a set 
of trapezium rule contributions in terms of central differences at the beginning 
and end of the range o.f integration. If the only available values of the integrand 
are those from/^ to the ends of the range of integration, then only forward 
differences from the beginning and backward differences from the end of the range 
are available, and a formula in terms of these differences is needed. This is 

an 

+ + (6.37) 

and is known as Gregory ’5 formula. 

Its practical use is limited, because usually the reason for the limitation on the 
range over which a function is defined is the occurrence of a singularity, as at 
1 

a: = i 1 in J e'^^’^{\ — x^)~^ dxy in which case the extension of the integration 
-1 

formula up to those points is invalid. In most other cases, values of the integrand 
outside the range of integration, and therefore the central differences required in 
formula (6,20), are available, and then this formula should always be used in 
preference to Gregory’s formula.! 

6.52. Integral in terms of function values 

By expressing the differences in formula (6.20), (6.21), or (6.37) in terms of func¬ 
tion values, we obtain a set of formulae expressing the integral as a sum of multiples 
of the values of the integrand and, in the case of formula (6.21), its derivatives. 
Such a formula is sometimes called a ‘Lagrange-t^qDe’ integration formula, by 
analogy with the form of Lagrange’s interpolation formula. 

For example, if in (6.20) we substitute 

/^S/o = i(/i—/_i)> P'S/yi = i(/n+i—/n-i) 

and neglect terms beyond these, we obtain 

Xn 

J/(.t) dx = (Sa^)[—A/_i + i/o + M/i+/2+"-+/n_2+l4/n_i + i/n~ 2 \/n 4 -l]- 

(6.38) 

Coefficients in a number of formulae of this tj-pe, differing in the order of differences 
to which they are correct and in the number of function values outside the range 
Xq to x„ used, are given in Chambers's 6-Figure Tables.% 

t For an example of the great advantage of a central-difference formula over Gregory’s 
formula, see Chambers's 6-Figure Tables (1949), vol. 2, pp. xxxiv and 548, or Chambers's 
Shorter 6-Figure Tables (1950), p. xxv. (The central-difference foi-mula is there called 
Gauss’s formula.) 
t Vol. 2, p. 549. 
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The writer’s own preference is for formulae in terms of differences, such as (6.17), 
as it is much easier to see which differences have to be taken into account, and 
inclusion of an extra one does not mean altering the whole formula. 

6*53. Use of Simpson’s or Weddle’s rules 

The 8y terms in the integration formula (6.23) can be taken into account by using 
Simpson’s rule for the intervals Sx taken in pairs, instead of by using the term 
(6.20). There is no great advantage in this procedure, except the 
smaller coefficient of the Sy term, and it has certain mild disadvantages. It is 
equivalent to calculating a correction to the trapezoidal formula for each pair of 
intervals in the integration, which is unnecessary when only a single value of an 
integral between fixed limits is required, and it involves a substantial amount of 
work in calculating a correction which may vanish identically, as, for example, in 

oo 

I dx. It also gives different weights to alternate function values, and requires 
0 

that the total number of intervals required to cover the range of x should be even. 
Use of Weddle’s rule has similar disadvantages, of which the fact that the number 
of intervals has to be a multiple of 6 may be more serious. 


6.54. Integrals of functions for which = 0 at both ends 

of the range of integration 

b 

If all odd derivatives of f{x) are zero at one of the limits of J f{x) dx, then 

a 

the total contribution from that end of the range of integration to the correction 
to the trapezium rule formula, as expressed by the scries of which the first few 
terms are given by the Euler-Maclaurin formula (6.22), is zero. And if all the odd 
derivatives are zero at both ends of the range, it would appear from this that 
the expression for the integral as the sum of a number of trapezoidal contributions 
is exact, whatever interval length (Sx) is used in the integration. 

TT 

An example is provided by the integral (I/ 77 ) J cos(2/sin0) for the Bessel 

0 

function J^iy)^ Here the integrand is an even function both of d and of (tt— 0), 
so y( 2 n+i)( 0 ) ^ ^ 0 fop ap and for integration intervals S0 = ttJN 

{N integral), the Euler-Maclaurin formula might appear to give 


(I/77) J cos( 2 /sin 0 ) = (l/2JV)j^l + 2 2 cos(ysin(m77/A^)}j (6.39) 

ex€ictly, for any value of N, Evaluation of the right-hand side of (6.39) for y — 2 
and different values of N gives the following results, to the accuracy of eight- 
decimal tables; 

N-\ -I 


N 

80 

(l/22\/')[l + 2^ 
L m 

12 

77/12 

0-22389 078 

8 

77/8 

078 

6 

77/6 

079 

4 

77/4 

0-22393 6 

3 

77/3 

0-22630 

2 

77/2 

0-29193 
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The value of the integral is Jq{2) = 0*22389 0779. Thus the relation (6.39) is 
nearly satisfied for values of S0 which are quite considerable, but it is not exaet; 
moreover the error increases with with drastic rapidity when is greater than 
about 0*6 radians. 

One reason why formula (6.39) is not exact is this. Just as Taylor’s series to 
m terms has a remainder after m terms, which remainder may not tend to zero 
as m -> 00, so has the Euler-Maclaurin formula (6.22). If the odd derivatives of the 
integrand vanish at both ends of the range of integration, the remainder after m 
terms is independent of m; but it does not follow that it is zero. A similar situation 
may oecur with Taylor’s series. For example, it can be shown (most easily by 
induction) that for the function g{x) defined by 

g{x) = 0, X = 0, 

the mth derivative at the origin, gr^^)(0), exists and has the value zero for all values 
of m; hence for every finite value of x, every term in the Maclaurin series for this 
function is zero, so the series converges to the value zero. Nevertheless, g{x) is 
not zero for any value of x except x = 0 (though it is very small for small values 
of x). Similarly, the error of a trapezium-rule formula, for a given value of Sx, 
may be non-zero although all the terms involving in. the formula 

for the error may be zero. This error will tend to zero as Sx -> 0, but this is not the 
point here; the point is how the error, for a given value of Sx, depends on the 
number m of‘correetion’ terms taken in the Euler-Maclaurin formula. A similar 

27r 

example is provided by the integrals J /(0)cos 2Tmd dd occurring in the harmonic 

0 

analysis of a periodic function (see § 11.2). 

00 

Another example is provided by the integral J dx. In this case the integrand 

0 

is an even function of x, so every odd derivative is zero at x = 0; also all derivatives 
are zero at x = oo. Hence the Euler-Maclaurin formula appears to give 

J dx = (8a;)f (6.40) 

exactly, for any value of Sx. If both sides of (6.40) are evaluated for different 
values of Sx, the results are as follows; 


J 6“*^® dx = 

0 

Sx 

0*5 

0*6 

0*7 

0*8 

0*9 

1*0 

1*1 


0*88622 69254 5 to eleven decimals 



2 

i = i 


0*88622 69254 5 to eleven decimals 
69254 8 
69285 
0*88622 72808 
23 598 
32 0 

0*88674 


Thus the relation (6.40) is nearly true for values of Sx which are quite considerable, 
but, like (6.39), it is not exact, and for the same reason. 
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b 

Another aspect of this behaviour of the error of the trapezium rule for J f{x) dx 

a 

wheny( 2 ^+i)( 6 )_y( 2 w+i)^Qi j = 0 is that the Euler-Maclaurin formula is only asymp¬ 
totic. We saw in § 4.6 that the finite difference operators, and Taylor’s series in 
the form E = could be used freely on functions which were the products of 
polynomials and exponentials of linear functions of x; is not of this form, and 
an examination of the error term is necessary before the Euler-Maclaurin formula 
is applied to it with 8x so large that the approximation to the integrand in each 
interval by a sum of products of polynomials and exponentials becomes dubious. 
Such an examination has been carried out by Goodwin, f 

The point might seem of formal rather than practical interest, since anyone 
with experience of numerical work, faced with the values of at intervals of, 
say, Sx = 1, namely: 


X 

/ = 

8/ 

8y 

sy 

sy 

0 

1 

— 6321 

— 12642 

+ 15467 

+ 30934 

1 

0-3679 

-3496 

+ 2825 

+ 489 

-14978 

2 

0-0183 

— 182 

3314 

- 3133 

— 3622 

3 

0-0001 

— 1 

181 

- 180 

+ 2953 

4 

0-0000 


1 

- 1 

179 

5 

0-0000 




1 


would say that these function values alone did not define the integrand well enough 
to justify evaluating the integral to more than two significant figures at most; it is 
hardly necessary actually to form the difference table to reach this conclusion. 

CO 

However, the fact that the integral J dx has this property raises the question 

0 

whether the use of relatively large intervals Sx in the evaluation of other integrals 

CO 

of the form J dx may also give results of useful accuracy. This also has 

0 

been examined by Goodwin.f 

Further light is thrown on this by considering the finite difference integration 
formulae (6.20), (6.21) in terms of the cardinal function (§ 5.91). It has been 
mentioned in § 6.2 (following formula (6.3)) that the coefficients in the integration 
formula (6.2) are the integrals of corresponding coefficients in Bessel’s interpolation 
formula. Let these be i 

= / B,n{e) dd. 

0 

Now i{f{x) is finite for all x, and tends to zero, as x -> co, fast enough for the series 
(5.41) for the cardinal function to converge, then the function whose intermediate 
values, between the tabular values f{xj), are given by Bessel’s interpolation formula 
is the cardinal function associated with these tabular values, so this cardinal 
function is the function whose integral is given by the infinite series 

i(8:r)[/o+/i+ J/2n(8^"/o + 8^"/l)]- 

t E. T. Goodwin, Proc. Comb. Phil. Soc. 45 (1949), 241. 
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And it can be verified from the definition of the appropriate cardinal function 
that its integral is given by the expression on the right-hand side of expression (6.40). 

6.55. Evaluation of a definite integral when the integrand has a 
singularity 

In evaluating an integral J f{z) dx of which the integrand has a singu¬ 
larity, the singularity can often be removed by a change of independent 
variable. If the integral is required as a function of the upper limit, we 
may want to avoid this in order to obtain directly, without further 
interpolation, values of the integral at equally spaced values of x. But 
this does not apply to an integral between fixed limits, and in this 
context the only reason for avoiding a change of variable is that a certain 
amount of interpolation may be required in order to obtain the values 
of the integrand at equal spacings in the new variable. 

There is, of course, no need to use the new independent variable over 

1 

the whole range. For example, to evaluate J [f{x)j{l—x)^ dx we might 

0 

use the substitution x= 1—over the w^hole range of x, and so evaluate 
the integral as ^ j 

0 0 

or we might divide the range of x into two parts, one from a: = 0 to ^ 
and the other fi’om z = ^ to 1 and only make the substitution in the 
second part, thus evaluating the integral in the form 
1 i ''( 1 -^) 

0 0 0 

In this case ^ should be chosen so that 1 —^ is a perfect square. For 
example ^ = 0-64 might be taken; this would enable intervals of 0-04 
or 0-08 in x to be used in the first integral and intervals of 0-1 in y in 
the second. 


6.56. Definite integrals which are functions of a parameter 

An important class of integrals between fixed limits comprises those 
which define a function of a parameter which occurs in the integrand. 


such as 


1 

-1 


b 

or generally g{x) ~ J f{x, u) du. 

a 


(6.41) 


Such an integral can be evaluated by quadrature for each value of x, 
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and this may be the only way of evaluating it. But another method of 
treatment may be much easier and less laborious if it is possible at all. 
This consists of finding a differential equation which the integral (6.41) 
satisfies, and solving this differential equation by a numerical process 
(see Chapter VII). The amount of work required to obtain a single 
value of the integral is then very much less than that required to carry 
out the evaluation by quadrature, and probably evaluation by quadra¬ 
ture will only be carried out for two or three values of x, to give initial 
values for the integration and to provide an overall check. It is not 
always possible to obtain such a differential equation, but many integrals 
of this kind of which the values are actually wanted in various contexts 
do satisfy differential equations. In such cases, the differential equations 
can often be obtained by one or two differentiations with respect to x 
and an integration by parts with respect to u. 

Consider, for example, the function 

00 

/(^) = / [e"“V(«+^)] du, 

0 

which has been studied by Goodwin and Staton, f The range of integration here is 
infinite, but for x > 0 the integrand is of such a form that differentiation with 
respect to x is justified. One differentiation gives 

00 

f\x) = — J 
0 

and integration by parts with respect to u then gives 
00 

/'(x) = J 

0 

00 

= [e-“V(«-t-a;)]”_o+ / 2«[e-“*/(«-t-a;)] da 
0 

00 00 

= — (l/a;) + 2 J e““* da — a; J 

■•o 0 

= -(I/X) ^7T^-2xf(x), 

so that this function f{x) satisfies the differential equation 

f^+2xf = 7 T^ — (l/x), (6-42) 

Evaluation off{x) by quadrature for one value of x, say x = 1, could be used to 
give a value from which the numerical integration of equation (6.42) could be 
started, though Goodwin and Staton actually used a series expansion to obtain 
such a value. 

f Quart, J. Mech. and Applied Math, 1 (1948), 319. 
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6.6. Use of unequal intervals of the independent variables 

As already mentioned, in evaluating integrals between fixed limits 
there is no need to use values of the integrand at equal intervals in x, 
and there may be advantages in using formulae in terms of some other 
set of values. An integration formula using a finite number of values 
of the integrand can be regarded as giving a weighted mean of these 
values: ^ 

J f{x) dx = (6-a)[2 (6.43) 

where the w^.’s are the weights assigned to the values of the integrand 
at the points x,^. Given any («-+!) points X/^, not necessarily equally 
spaced, values of w,^ can be obtained which will make such a formula 
correct for any polynomial of degree up to n. And it is possible to put 
a condition on the W/^’s (such as that they should all be equal) and 
determine the corresponding Xj^’s such that this formula should be exact 
for polynomials of degree up to n. But if no condition is imposed on 
either the x^.’s or the then these can be determined so that formula 
(6.43) with (w+1) terms in the sum will be exact for any polynomial of 
degree (2n.-|-l). Such a formula is known as an (n+l)-point Gaussian 
integration formula. 

6.61. Gaussian integration formulae 

The values x^ of the independent variable, at which the integrand 
values in a Gaussian formula are to be taken, and the weights to be 
assigned to them in formula (6.43), can be found as follows. 

By the transformation 

^ = [2x-{a+b)]l{b-a) 

the range of integration is reduced to the range ^ = — 1 to +1. Let P„(^) 
be the Legendre polynomial of degree n; these polynomials have the 
property ^ 

J -Pm(^)-Pn(^) di ^ 0 \im ^n, 

-1 

which is expressed by calling two such polynomials, of different degree, 
‘orthogonal’ over the range | = — 1 to +1. 

Any polynomial of degree (2n+l), say he expressed as 

P2n + l{^) — (6.44) 

(IrM) being the quotient and remainder polynomials on division 
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by -fn+i(^) > 9'n(^) is a polynomial of degree n and of degree n 

at most. Then 


/ = I A.itf )?„«) di + f ••„(« (6.45) 

-1 -1 -T 

Now since qj^^) is a polynomial of degree n, it can be expressed as a linear 
combination of Legendre polynomials with m ^n; each of these 
is orthogonal to Pn+i(^)> hence 

1 

/ dl = 0, (6.46) 


-1 


whatever the quotient polynomial and (6.45) becomes 

1 1 

J Pzn^M) dS = l rji) di; (6.47) 

-1 -1 

the vanishing of the integral (6.46) through the orthogonal property of 
the Legendre polynomials is the reason for taking the Legendre poly¬ 
nomial as the divisor in (6.44). 

Now we want to find values of such that 
1 

f Pzn+lii) = 2 2 i^’kP2n+l(ik)> 

-1 * 

that is, on substitution from (6.44), 

1 

/ J)2,.i(^) = 2 2 (6.48) 

-•1 * 

The result (6.47) shows that the integral on the left-hand side of (6.48) is 
independent of the quotient polynomial g'„(|), and the expression on the 
right-hand side of (6.48) can be made independent of g'„(^) if (and only 
if) the i/.’s are taken as the roots of the polynomial equation 

A+i(fl = 0. (6.49) 

These roots are all real and distinct, and lie in the range — 1 <^< 1; 
they give the values of ^ at which the values of the integrand are to 
be taken. 

Since rj^(^) is a polynomial of degree n, it is determined completely 
by its values for (ri-fl) distinct values of Let these be taken as the 
(w-]-l) roots (1 ^ ^ ^ ^+1) of equation (6.49), and let J^„+i(^) be 
the function 




(6.50) 
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Then from the expansion of in partial fractions (or from 

the equivalent Lagrange interpolation formula, see § 5.7), 


-•««)= 2 








(6.51) 


This formula is exact, since r„(^) is a polynomial of degree w, and inte¬ 
gration gives 

-1 -1 

Now from formulae (6.47), (6.48), the weights Wf^ are given by 


f ^n(^) = 2 <^fc^n(4). 

-1 *= 


and comparison of this with (6.52) gives for the weights the values 

U, =1 f 

2 J i'n+l{^k)ii 4) 

-1 

Values of W/^ and 4 for Gaussian quadrature formulae up to r- == 16 
have been tabulated by Lowan, Davids, and Levenson.f The limits of 
integration can be reduced to 0 and 1 , instead of —1 and 1 , by the 
transformation 17 = {x—a)l{b—a) = f(l-f 4 ^); values of r)/^ and are 
given by Whittaker and Robinson. J 


0-8 

Example: To evaluate J e+** cte by a five-point Gauss formula. The values of 

0 

XVj, Xj, and the integrand values are as follows: 


y 

6 

XVj 


II 

® 

1 

0046910 

0 118464 

0-037528 

1 001409 

2 

•230765 

•239314 

•184612 

1 034669 

3 

•5 

•284444 

•400000 

1173511 

4 

•769235 

•239314 

•615388 

1-460388 

6 

0-953090 

0 118464 

0-762472 

1-788475 



= 1-261401 

(6—o) = 0-8 



(6-a) 2 Wjfj 

= 1-009121 




Note: This value of the integral agrees with those calculated in §§ 6.4 and 6.5. 

For work on numerical evaluation of integrals of given functions, the 
practical value of a Gaussian formula such as (6.43) is limited by two 

f Bull. Amer. Math. Soc. 48 (1942), 739. See also Kopal, Numerical Analysis (Chap¬ 
man and Hall, 1955), appendix iv, § 4.1. 

J Calculus of Observations (Blackie, 4th ed., 1944), § 80. 
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things: first, the interpolation which may be necessary to obtain the 
values of the integrand at the required values of x^., and secondly the 
difficulty of checking adequately both these values, which are at unequal 
intervals in x, and the evaluation of the integral from them. Further, 
if the integrand is known at equal intervals of x, as will often happen, 
no advantage is taken of this. When the integrand is given by a formula 
sufficiently simple for its value to be calculated from this formula for 
each value of Xj^, the need for interpolation does not arise, and in such 
a context use of a Gaussian formula may be a practicable and useful 
process. A check can be provided by carrying out two independent 
integrations, say one with an w-point formula and the other with an 
(n-l-2)-point formula. 

Use of a Gaussian integration formula may also be very valuable in 
simplifying problems in more than one variable. It can be used, for 
example, to simplify integro-differential equations involving integrals 

TT 

of the type J /(r, 0)sin d dd. If such an integral is replaced by a sum 

2 'w{dj)f{r, 6j), the solution of the equation in which the integral occurs 
i 

can be reduced to the solution of a finite number of equations for the 
functions f{r, dj), each of which is a function of the single variable r 
only. In making such a replacement it is clearly desirable to obtain as 
good an approximation as possible with a small number of terms, and 
this is obtained by taking the values of 6^ and the weights w{6j-) to be 
those of a Gauss formula.f 

x> 

6.62. Gaussian formula for J e-’^^p 2 n+t{^) dx 

0 

The argument of the previous section will not apply to an integral over an infinite 
range. However, a similar argument can be applied to integrals of the t 5 rpe 

00 00 

J dx and J e-^^'p 2 n+i(x) dx, where P 2 ,i+i(x) is a polynomial of degree 

0 0 

(2n-f-1) in a;, and the results can then be applied to the approximate evaluation 
of infinite integrals in which the integrand, though not exactly of the form 
e~^^P 2 „+i(x) or e~^^'p 2 n+i(x), is known or believed to be approximately of one of 
these forms. The coefficient k in the exponential factor can be removed by a 
change of scale of the independent variables; we shall consider as an example the 
00 

integral J dx, 

0 

The argument leading to formulae (6.49), (6.52) of the previous section depended 

I See, for example, S. Chandrasekhar, Astrophys, Journ. 100 (1944), 76, and Radiative 
Transjer (Clarendon Press, 1960), §§ 20, 25; also G. C. Wick, Phys, 121 (1943), 702. 
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on the use of the orthogonal property, of the Legendre polynomials to give the 
result (6.46), Correspondingly we now want to write 

Ptn+li^) = Lr,+i{x)qn{x)+rn(x), 

where the divisor polynomial L^i(x) is of degree (n+1) and is one of a set of 
polynomials such that 

00 

J e^*L„i(x)LJx) dx = 0 for m ^ n, (6.53) 

0 

that is, the functions e“^^L„(x) are orthogonal over the range a: = 0 to oo. Then, 

00 

corresponding to (6.46) we will have J e“^L^^i(x)g„(x) = 0, and consequently 

0 

00 00 

J dx, 

0 0 

corresponding to (6.47). Apart from the presence of the factor in this integrand, 

the rest of the argument follows that of § 6.61. 

The polynomials L^(x) with the orthogonal property (6.63) are those known as 
the Laguerre polynomials, defined by 

= e+==(^)"(x™e-='); 


for an n-point formula (correct for polynomials 7 ? 2 n+i(^))» values of the Xj^'s 

are the roots of r / \ a 

^n+i(^) = 0, 


and if 


— {x X2)**>{X Xyj_|_j), 


the weights are 




J 


^n+i(a^)g~"' dx 
K+i(^k)(^—^k)' 


Values of x^, for n-point formula up to n = 16 have been calculated by Salzer 
and Zucker.f 

00 

In applying these results to the approximate evaluation of integrals J / (x) dx 

0 

in which the formal behaviour of the integrand is not precisely known, it is neces¬ 
sary to take a factor out of the integrand f{x), and the result may depend 
on the value of k adopted. For this reason, some discretion is required in such 
a context. 


6.7. Numerical differentiation 

We have already seen that a table of values does not define a function 
uniquely. Still less does it establish whether the function tabulated is 
differentiable everywhere, or even anywhere, within the range of the 
table; two functions may be indistinguishable, to any specified degree 
of numerical accuracy, for every value of x (not only for the tabular 

t Bull. Amer. Math. Soc. 55 (1949), 1004. See also Z. Kopal, Numerical Analysis 
(Chapman and Hall, 1955), appendix iv, § 4.2. Tables of abscissae and weights for a 
number of other integration formulae of Gauss type are also given by Kopal. 
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values), yet one may be differentiable everywhere and the other nowhere. 
And still less does a table establish whether a function is differentiable 
two or more times. These considerations alone suggest that numerical 
differentiation of a function specified by a table may be a dubious process. 





Further, the entries in a table are affected in an irregular way by 
rounding errors, and differentiation exaggerates irregularities whereas 
integration smooths them. In differentiation we are concerned with a 
limit process carried out on the quantity [f{x-\-Sx)—f{x)]j{Sx), and as 
Sx becomes smaller, irregularities in the values of / become more pro¬ 
minent in the result; whereas in integration we are concerned with a 
limit process carried out on the quantity ^fSx and the effect of an 
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irregularity in any one value of / becomes less prominent a^ 8x becomes 
smaller. This is illustrated by Fig. 10 , which shows the graph of the 
function 


y = 7: 


1 sin \0ttx 


-lOOx* 


2 IOttx 


which might represent approximately an irregularity produced by a 
rounding error at a: = 0 in a table at intervals of 0-1 in x, and the deriva¬ 
tive and integral of this function. 

For such reasons, the differentiation of a function specified only by 
a table of values, or determined experimentally and so subject to errors 
of observation, is a notoriously unsatisfactory process, particularly if 
higher derivatives than the first are required. It is a process to be 
avoided if possible, unless the context in which the results are required 
is such that the limited accuracy attainable by the numerical process 
is certainly adequate. In general, values of the second and higher 
derivatives obtained from such data should be regarded with caution 
if not scepticism. 

In some cases it may be possible to evaluate derivatives by some pro¬ 
cess other than numerical differentiation. If, for example, a function y 
is known to satisfy a first-order differential equation, values of y' can be 
obtained by substituting the values of y into the differential equation. 
And if y satisfies a differential equation of higher order, it is usually 
better to obtain y' by numerical solution of the equation as an equation 
for y', taking y as a given function of x, than to try to obtain y' directly 
from the values of y by a differentiation formula. 


6.71. Differentiation formulae 

To obtain a first-order derivative there are two useful formulae, one 
giving the values of the derivative f at the values x^ of x at which the 
function is tabulated, and the other giving/' at The former has 
already been obtained in §4.71, and is (see formula (4.46)) 

(6.54) 

The other can be obtained either by differentiating Bessel’s interpola¬ 
tion formula with respect to p and then putting p = or by using finite- 
difference operators as follows. We want to find a /( 8 ) such that 

{hx)f\ = <^(8)8/i. 

Hence 

,^(8) = C7/S = (3inli-4S)/iS = l-i8«+j|»8‘-A5‘+0(8a:)" 
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on putting n = 1 in (4.42). Hence 

(6.55) 

This formula is much preferable to (6.54) on account of the more rapid 
decrease of the coefficients of the higher orders of differences. If, 
however, values of /'• are required, there is no advantage in using 
(6.55) followed by ‘half-way’ interpolation between the values of/)+j 
by use of formula (5.3), since these two processes together just give 
formula (6.54). 

For a second-order derivative, the appropriate formula is (4.43), 
namely ^ (0.56) 

In carrying out the calculations, the interval (8a:) taken should not he 
too small, since the smaller it is taken, the smaller the number of signifi¬ 
cant figures in 8/^ and so in/' (and similarly for a second derivative). 
Rather, 8a: should be taken as large as is convenient, subject to the 
truncation error of the differentiation formula used being negligible.f 

It will often be advisable either to smooth the values of / before 
differentiation, or to smooth the values of/' or/" obtained (for a smooth¬ 
ing process see §11.4). Let us write f'^ for the smoothed values which form 
an approximation to/'. To ensure that no systematic errors are intro¬ 
duced in the smoothing process, the values of /' should be integrated, 
and compared with the original values of/. If the quantities /— J/^ dx, 
which are called the ‘residuals ’, show any significant systematic variation 
with X, a process of differentiation should be carried out on these residuals. 

Example: The function tabulated below as Y{x) is the solution of the equation 
y" = I -\-xy with 2 /( 0 ) = 2/'(0) = 0, and the function z{x) is the solution of z" = xz 
with z(0) = 0, z'(0) = 1; to find y'{0) and y{2) for the solution of y" = l+xy for 
which 2 /( 0 ) = 0, 2 /'( 2 ) = 0. 


X 

Y(x) 

8=*r 

8^r 

8«r 

z{x) 

1-6 

1-56205 





1-7 

1-83254 

4125 




1-8 

2-14.428 

4873 

154 


2-80444 

1-9 

2-50475 

5775 

188 

7 

3-17749 

2-0 

2-92297 

6865 

229 

7 

3-61107 

2-1 

3-40984 

8184 

277 

18 

4-11708 

2-2 

3-97855 

9780 

343 


4-70978 

2-3 

4-64506 

11719 




2-4 

5-42876 






z'(2-0) 

= 4-67626 


t For a closer analysis of the best interval to use in numerical differentiation, see 
Z. Kopal, Numerical Analysis (Chapman and Hall, 1966), § III-E. 
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Two standard solutions of the equation y'' = xy have been tabulated.f They 
are written Ai(:r) and Bi(a:), and the function z{x) of this example is related to 

them by ^ _ 3iAi(x)]/2.3*^, 

where jS is a constant given in the Introduction to the Tables (p. B. 17; the value 
of 2. is 0*896577; this function z{x) is that written y^i^) in that Introduction). 
The values of z{x) and z\x) here tabulated have been calculated from this formula; 
only the value of 2 ( 2 * 0 ) is required to give the results sought, but neighbouring 
values are given for use in a check. 

The general solution of y*' = \-\-xy with y = 0 at a: = 0 is 

y = y+C2, 

where c is an arbitrary constant; for the solution with y\2*0) = 0, 

c = -y^( 2 * 0 )/ 2 '( 2 * 0 ), 

so we need to determine Y\2*0), From the tabulated results and formula (6.43) 

0*2r^(2*0) = 0*90509-i(2409)+^(89)-rio(ll) 

= 0*90509-40l5 + 3o-Oi = 0*901104, 

while 0*22'(2*0) = 0*936252, so c = -(0*901104)/(0*936252) = -0*963482. 

With this value of c we have 


X 

Y -cz 

y 

8*2/ 

8*y 

1-8 

2-14428-2-70203 = 

-0-55775 



1-9 

2-50475-306145 = 

-0-55670 

— 58 


2-0 

2-92297-3-47920 = 

-0-55623 

-113 

-1 

2-1 

3-40984-3-96673 = 

-0-55689 

-169 


2-2 

3-97855-4-53779 = 

-0-55924 




and for this function y, 2/(2*!)—2/(1 *9) = —0*00019 and 8^2/(2*l)—8^2/(l*9) = — 111, 
so 0*22/'(2*0) = -0*00019—K-lll)+^(~l) 

= -0*00019+185 = -O-OOOOOg, 

which is within the tolerance for rounding errors. This provides a check of the 
work. 

Notes-, (i) The value of y{2'Q) is not determined correctly to a imit in the last 
figure; the value c = —0*963475 gives 0 * 22 /'( 2 * 0 ) = + 0 * 000005 , which is equally 
within the tolerance for rounding errors, and 2/(2*0) —0*55621. 

(ii) The value of y\0) for this solution is just y\0) = c. 

(iii) The value of F'(2*0) is not determined with certainty to several units in the 
fifth decimal, since 2/(2*l) — 2 /( 1*9) is subject to rounding errors up to 1 in the fifth 
decimal and is multiplied by l/2(8x) = 5. A more accurate value of F'(2*0) 
could be obtained by using 8x = 0*2, but the contribution to 2(8x)Y' from the 
higher orders of differences would be considerably greater; that from 8^Y, for 
example, would be greater by a factor of over 100. 


j* British Association Mathematical Tables, Part-volume B (1946), The Airy Integral. 
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6.72. Graphical differentiation 

The residuals f — j f'^dx of a numerical differentiation over a range 
of X will usually be numbers of one or two digits only, so can easily be 
plotted to the accuracy to which they are known. In such a case, a 
graphical method of carrying out the differentiation is adequate. The 
best way of doing this is to plot on good squared paper (see § 2.5) the 
values of the function to be differentiated, and through each plotted 
point draw a vertical line to indicate the range of uncertainty, due to 
rounding error or other causes, of that value. Then draw the smoothest 
curve passing each plotted point within the indicated tolerance. The 
latitude in drawing such a curve will give an indication of the rehability 
of the values of the derivative. 

The best way of finding the gradient of such a curve, or of one repre¬ 
senting a set of results of some experiment or observations, is as follows. 
Take a flat piece of polished sheet metal (aluminium or stainless steel 
is satisfactory), or surface-aluminized glass, mounted in such a way 
that it can be placed on a piece of paper with its surface accurately 
perpendicular to the paper and extending right 
down to the paper. Set this so as to intersect 
the curve at the point at which the gradient is 
wanted (see Fig. 11), and rotate it until there is 
no discontinuity in direction between the curve 
and its reflection in the mirror. With care, this 
setting can be made with considerable accuracy, 
probably greater than that to which the curve 
can be drawn. The gradient of the curve can 
then be determined directly from the inter¬ 
sections of the plane of the mirror with the grid 
lines of the paper in which the curve is plotted. 

6.8. Errors of interpolation and integration formulae 

W. E. Milnef has given a convenient general method for obtaining 
formally exact expressions for the truncation errors of formulae for 
interpolation, integration, etc. 

We have derived and e^ipressed such formulae as results of linear opera¬ 
tions on the function to be interpolated, integrated, or differentiated. 
If we take one of these formulae to a finite number of terms, then the 
remainder after n terms can also be expressed as the result of a linear 
operation on this function. For example, if we take formula (6.11) as 

■j- Numerical Calculua (Univ. of Princeton Press, 1949), §§ 30, 31. 

K 



6013 
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far as the terms in /', the remainder, which is the truncation error of 
the formula in this form, is 

Xo 

= (8:c)[i(i;-l)-i(£+l)+A(i;-l)C7j/„. (6.57) 

which is the result of a linear operation R on /. Milne calls an operator 
R ‘of degree n’ when Rx^ = 0 for m ^ n, Rx^+^ ^ 0, and writes 
for an operator of degree n. The purpose is to obtain an expression 
for R^f for any function /. It is assumed that R^ does not involve 
higher powers of U than U^~^. 

R may contain some shift operators E so that Rf^ may depend on 
values of/ or its derivatives for values of x other than Let be the 
least and Xj^^ the greatest of these; and let a be a value of x less than x^. 
Also let us write <f>n{z) for the function 

= 2" for z ^ 0, ^ for z < 0. (6.58) 

One form of Taylor’s series to n-\-\ terms with a remainder is 

f(x) = }(a)+{x-a)r(a) + j,(.x-afna)+...+ 

X 

+ 1 (l-a)"/<“>(a) + i J H- (6.59) 

a 

this form can be obtained by repeated integration by parts, using 

X X 

a a 

The last term in (6.59) can be written 

00 

t f /<”+«(i)^„(:c-fl d(, (6.60) 

a 

since the integrand here is zero for ^ > x. 

The first n+1 terms in (6.59) form a pol^momial of degree n, so they 
are annihilated by the operator R^. Also since R^^ operates on functions 
in so far as they are functions of x, it only operates on the function 
in the integral in the form (6.60); this is the reason for expressing the 
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integral in this form, in which the limits are independent of x. Hence 

CO 

•R./W = ^ J di. 

a 

Since a has been chosen to be smaller than the smallest argument 
occurring in it follows that the arguments of aU the terms in 
^re positive for ^ < a; but 0 „( 2 ) = for positive 2 , so 
= 0 for I < a. So the lower limit of the integral can be 
replaced by — 00 , and finally 




( 6 . 61 ) 


where 


<?(!) = 


n[ 


(6.62) 


If R^f{x) is a function of x, then G{$) is a function of x as well as of 
if E^f{x) is not a function of x, as is the case for the operator in (6.57), 
then G{^) is not a function of x. 

The function G{^) consists of polynomial segments between the 
values of Xj involved in R^f{x), and is zero outside the range of these 
values. It is also independent of the function on which R^ operates; 

hence \RJ(x)\ < Ji:.[max|/(»+«(i)| in < x„]. 


where 


K is independent of the function / on which R operates. 

In many cases G{^) is of constant sign over the range where it is not 
zero, and then a better formula for the error can be obtained. The mean 
value theorem, applied to (6.61), then gives 


CO 

I «(f) di. 


(6.63) 


where < X < the integrand of (6.61) is zero outside these limits 
so X must lie in this range. Also for f{x) = x^+^/{n-{-l)\, = 1 

everywhere, so that in this case (6.63) gives 


uu 

J G{^)di = R^x-^^jin+l)l 


and hence, in general, 

Rnfi^) =f^^^\X)R,x-+y{n+l)l 


(6.64) 
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As Milne points out, the evaluation of on the right-hand side of 

(6.64) is often much easier than the determination of the pol 3 momial 
segments of G{^) and their integration. In (6.64), can be replaced 

by J2„(a:—6)”+^ for any constant b if this is more convenient for the 
evaluation of this quantity. 

Example: To obtain a formula for the error of trapezium rule integration 

]fdx = i(8a;)[/o-l-/,]. 

Here Ef{x) = jfdx 

Xo 

which is identically zero forf(x) = x, but not foTf{x) = x^. Hence i? is an R^, and 

Xi 

Ri<l>i(z-i) = j dx -U^x)[^i(Xo-i)+(f>i{x^—i)]. 

xa 

Forf < a;o,x—dispositive over the whole range X = Xq to Xi,so^i(x —d) = 
for all relevant d» and 

which is zero, as it should be. For < d < — = 0 for x < d> so 

ii(?i(d) = Rii>,{x-i) = = i(^i-d)(^o-f). 

Ford > ^>^i(a:—d)iszerooverthe whole rangeofx;so (?i(d) = 0. Hence altogether 
Giii) = ) for Xo < d < 

= 0 otherwise. 

Hence for a general function/(x), 

Rif=ijr(i)(x,-i){x,-i)di. 

Xo 

In this case G(d) is zero or negative, and is of so simple a form that it is easy to 
00 

evaluate J (?(^) directly. Substitution of ^ gives 

— 00 

00 1 

/ G(i)di = i J J -rjil — rj) dr) = —^(8a;)®. 

— 00 Xo 0 

Alternatively, taking/(x) = {x—Xf,Yj2\ and using (6.64) we have 
i?i(a:-a;o)V2! = i[J(a;i-a;o)«-^^(8a;)(a;?H-a;o)] 

= A( 8 a:)[ 2 (a;?-fa;ia;o+ 4 )— 3 (a:J-l-a;o)] 

= 1*2(8^:)®, 

so that f?i/ = — i^/*(-X^)(8a;)®, where a:,, < X < Xj. 

6.81. Use of formulae for the error 

If a formula for interpolation, integration, etc., is such that the 
operator R of the previous section is of degree n, the error of the formula 
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involves the (7i.-l-l)th derivative of the function to which it is applied. 
But we have seen that for a function specified only by a table of values, 
the numerical determination of derivatives beyond the first or second 
is an unreliable process and one to be avoided if possible. Even when 
the function to which the interpolation or integration formula is to be 
applied is given by a formula which can be differentiated, the formulae 
for the higher derivatives may be too complicated to be convenient for 
numerical evaluation. Thus a formula which depends on the values of 
derivatives beyond the first or second is of limited practical use. 


VII 


INTEGRATION OF ORDINARY DIFFEREN¬ 
TIAL EQUATIONS 

7.1. Step-by-step methods 

One class of methods for the numerical integration of ordinary differen¬ 
tial equations consists of those in which the solution is evaluated step 
by step through a series of equal intervals in the independent variable, 
so that when the solution has been carried to x = the next step con¬ 
sists of evaluating the change in the solution through the interval 8x 
from Xj to In such a process we follow out in the course of the 

numerical work the development of the solution as the independent 
variable increases. For simple equations this can be made a straight¬ 
forward and easy process to carry out; it can be provided with adequate 
current checks to assure the worker that the integration is proceeding 
correctly, and in the writer’s experience it is one of the most satisfying 
forms of numerical work to carry out. 

7.11. One-point and two-point boundary conditions 

From the point of view of a step-by-step process, the nature of the 
boundary and other conditions to be satisfied is more important than 
the nature of the equation itself; and as regards the boundary conditions 
what matters is not what they are but where they are. If all the conditions 
which the solution must satisfy are boundary conditions given at one 
point of the range of integration (usually one end of it), the solution can 
be started from there with all relevant quantities known; and, apart 
from the possible occurrence of singularities or of instability in the pro¬ 
cess of integration, evaluation of a solution Usually gives no difficulty. 
Such conditions are known as ‘one-point’ boundary conditions, and 
a problem in which the conditions are of this type has been called 
by Richardson! a ‘marching problem’ as the solution is obtained by 
marching step by step from the initial data. 

Bui if some conditions are specified at one point, x = a, of the range 
and others at another, x = h (usually x = a, b will be the ends of the 
range) or if there is a relation between the behaviour of the solution at 
the two ends of the range such as a condition that the solution y should 
be periodic, which for a first-order equation is y{b) — y{a), or some 
t L. F. Richardson, Phil, Trans, Boy, Soc, 226 (1927), 300. 
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6 

integral condition on the solution as a whole such as | da; = 1, the 

a 

evaluation of the solution may not be so straightforward. Conditions 
which are specified at two points of the range are called ‘two-point’ 
boundary conditions; a set of conditions at more than two points is 
possible but unusual. A problem in which the conditions on the solution 
are not one-point boundary conditions Richardson has called a ‘jury 
problem’. 

If the equation, and .the conditions which the solution must satisfy, 
are linear, it may be possible to evaluate the solution as the sum of a 
particular integral satisfying the conditions at one point of the range, 
and a complementary function. But in many cases such a procedure is 
a formal possibility only and not a useful one for practical numerical 
work (see § 7.6); and if the equation is non-linear it is not available. 

A step-by-step solution has to start from some point of the range with 
definite numerical values of sufficient quantities to define a solution; 
for an nth-order equation these will usually be y and its first [n —1) 
derivatives, but they may be the values y^, y^, y^,---, Vn-i s-t the beginnings 
of the first n intervals. With one-point boundary conditions this point 
is naturally taken as the point from which to start the integration. With 
other conditions it is best to start from the point at which the values of 
the greatest number of values of y or its derivatives are specified by the 
given conditions on the solution. The other starting conditions have to 
be estimated and adjusted, either by trial or by the use of a comple¬ 
mentary function when this is practicable, until a solution satisfying the 
other conditions is obtained. 

We will consider first the step-by-step evaluation of a solution from 
given initial conditions, and later (§7.6) return to the consideration of 
the determination of solutions satisfying other conditions. 

7.2. Second-order equation with first derivative absent 

The simplest numerical process is that for a second-order equation 
with the first derivative absent 

y''=f{x,y), (7.1) 

in which f{x, y) need not be linear in y. This is integrated by using the 
formula for twofold integration from y" to y, without an intermediate 
calculation of y' (§ 6.44): 

2/i-2yo+2/-i = (7.2) 

One procedure will be explained first in some detail; there are several 
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variants of it, some of which will be mentioned later. It will be supposed 
that the term in (7.2) is neghgible, so that the integration formula 

is being used in the form 

8^0 = (8x)»[i/o'+A8Va. (’-3) 

Suppose the integration has reached x = Xq, and we are concerned 
with the integration through the interval Sx to a; = x^. At this stage 
we have and = f{X(^,yo) and the backward differences from these. 
The procedure is then as follows. Estimate h'^yl, and obtain an approxima¬ 
tion to § 2^0 from (7.3). Add this to to give an approximation to 
and add this to y^ to give an approximation to y^. From this calculate 
y1 and hence b'^yl — y'[ —%o+ 2 /'-i- ^ be the difference between this 

value of and that estimated. A change of the estimate of b^yl by e 
makes a change ^(Sa;)^^ in y^. If this is less than \ in the last figure 
retained in y, the estimate is adequate; if not, the estimate is revised and 
the calculation of the interval repeated; but the interval length ( 8 a:) 
should be taken so that this is seldom necessary. A convenient arrange¬ 
ment of the work is as follows: 

* y” 8 ^' y 8 * 2 / 



The quantities above the heavy lines are those which are known when 
the integration has reached a: = a: 0 ; the quantity h'^yl enclosed in a 
‘box’ is that which is estimated and if necessary adjusted, and the 
arrows show the sequence in which the various quantities are calculated. 

To start the integration, two values of y are required, and it is advisable 
to have three to provide a check and to give an indication of the values 
of These initial values will often be obtainable from a solution 

in series without requiring the evaluation of a large number of terms. 
In some cases it may be necessary to carry out a few steps of an integra¬ 
tion at a small interval before starting the main integration. 

Example-, y" = {l—x^)y, j/(0) = 0, y'{0) = 1, 8x = 01. y{0-l) and y(0-2) 

evaluated from the series solution: 


y = x+ix^—iix^+0{x'’). 
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X 

1 —X^ 

y" 




0-0 

1-00 

0 

992 

0 

-51 

0-1 

0-99 

-0992 


- 51 





941 


-53 

0-2 

0-96 

-1933^ 


- 104 




- 

"837 


-57 

0-3 

0-91 

-2770 

676 

-161 

-65 

0-4 

0-84 

-3446 

450 

-226 

-71 

0-5 

0-75 

-3896^ 

153 

-297 


0-6 

0-64 

-4049 





y 

(check) 

0 

10016^ 



•632628 


Her© the numbers above the broken line are obtained from the series solution. 
In this example, y is an odd function of x, so that we have S^ 2 /^'( 0 ) = 0 , as well as 
the value ( 0 - 1 ) = —0-0051. The value = 0-000988 is obtained in two 

ways, namely (a) from the first three values of y'^ and the integration formula (7.2), 
and ( 6 ) from the first three values of y. Agreement between these values forms a 
check on the starting conditions, and also checks that the h'^y" term in the integra¬ 
tion formula has been taken with the right sign; this term has the coefficient + 
here, whereas in the formula for a single integration, the first term in the correction 
to the trapezoidal formula has the coefficient — 1 \. This makes it rather easy to 
make a mistake of sign at this point, and it is as well to have a check that the 
right sign has been taken. 

If the integration has been taken to x = 0-5, the numbers above the full line 
will have been calculated; we will consider the integration through the next step, 
X = 0*5 to 0 - 6 . From the run of the third differences h'^y", the next value may be 
expected to be about —76, giving 8 ^ 2 /''( 0 ‘ 5 ) about —302; a twelfth of this is —25 

which gives S* 2 /( 0 - 5 ) = (0-01)(0-3896-0-0025) = 0-003871. 

This is entered in the h‘^y column and checked by forming the value of 8^[8^2/(f^’^)]» 
By operating on both sides of (7.3) with 8 ^, we have 

8^(S^2/)o == ( 8 ^)^M'+A 8 V?], (7.4) 

and use of this formula provides a good check on the values of hhj; it should be 
noted that the contribution from 8 ^ 2/0 ruay have to be included in (7.4) although 
it is negligible in (7.2). 

The value of 2 /( 0 - 6 ) is then built up from the value of 8^2/(^*^)> from it the 
value of 2 /"( 0 - 6 ) is obtained from the differential equation. This value is 

2/''(0-6) = 0-4049, 

whence 8 y (0-5) = — 297; the estimate was adequate and no recalculation of this 
interval is necessary. The calculation has reached the stag© from which we started, 
only one interval further on, and a similar calculation for the next interval can 
now be undertaken. 


The differences of the values of y" form a check against random mis¬ 
takes in these values, and use of formula (7.4) provides a current check 
of the values of 8^?/. The other process which needs checking is the 
twofold summation of these differences to give the solution y. 
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There are various ways of carrying out this check. If values of y have 
been built up by two successive summations, of ^hj to and of to y, 
as illustrated in this example, a good check is provided by evaluating 
the second differences of y by the method of § 4.45, which does not 
involve the calculation of first differences, and verifying that the values 
so obtained reproduce the values of 8^y. This check can be applied as 
each value of y is obtained, but is best carried out occasionally, say 
every ten intervals, in such a way as to verify the values obtained since 
the previous check. 

Another check, which can only be carried out on a series of values of 
y, is provided by taking a set of alternate values of y and differencing 
them to second differences, taking the corresponding values of y" and 
differencing as far as necessary for use in formula (7.2), and verifying 
that these values of 8^y, and of y" and its differences, do satisfy formula 
(7.2) with (8a;) equal to twice the integration interval. It will usually be 
necessary to use higher orders of differences in y” in this check than in 
the integration, but central differences of higher order than the second 
are available at this stage. 

Example: To check the solution of y" = (\ — x'^)y obtained above. Copying the 
values of y" and y at intervals 8 a; = 0 - 2 , and differencing them, we have the second 
to sixth columns in th© following table: 


X 

y" 

8y 8Y 

y 

8*2/ 

(8a;)*(2/''-b ^8*2/'—?4o 8^2/ *’ ) 

00 

0 

0 

0 



•2 

•1933 

-420 -70 

•20132o 

769, 

4(1933 - 350- 0,) = 759i 

•4 

•3446 

-910 

•410233 



•6 

•4049 






The last column gives the calculation of h^y{0-2), for 8 a; = 0-2, from the values of 
y" and its differences. Agreement to a unit in the sixth decimal with the value 
of 8 * 2 / is not to be expected; but this figure is only a guarding figure. 

The interval 8a: used, and the number of figures kept in the different 
parts of the calculation, will depend on the equation, the data occurring 
in it, and the accuracy required in the results. This example is representa¬ 
tive of the accuracy which it is convenient to keep in many calculations 
concerned with integration of equations occurring in scientific or tech¬ 
nical problems. The last figure in y in this example is a guarding figure 
only, and could well be omitted if the function f{x, y) in the equation 
involved some experimentally determined function which is not known 
to better than 1 part in 1000. In working to this accuracy, the first 
estimate of 8^y" in each interval, its division by 12, the addition of the 
result to y", can be done mentally; so can the multiphcation by (8a:)^ 
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when 8a; is a power of 10, as will often be the case. Then the first number 
written down is {y" or h^y, and the value of h^y is immediately 

checked by differencing. Such an integration can be carried out quite 
quickly. 

The smaller the interval 8a; taken, the better the estimate of 8^" 
can be made, and the smaller the quantity {hxY by which this is multi¬ 
plied. But it is not advisable to take very small intervals, first, because 
the amount of work required to cover a given range of x increases as 
the length of integration interval used decreases, and, secondly, because 
effects of rounding errors in the values of h'^y may accumulate rather 
rapidly in the double summation to give y. If a large number of small 
intervals are taken, additional guarding figures may have to be taken to 
ensure that the cumulative effects of rounding errors are negligible to 
the accuracy required in the final results, and this makes the amount of 
work involved increase rather more than proportionately to the number 
of intervals. The interval length 8x should therefore be taken, roughly 
speaking, about as large as is compatible with ease in the practical 
numerical working of the integration. As a rough working rule it should 
be taken so that for only about one interval in five does the calculation 
for an interval have to be repeated. If many have to be repeated the 
interval should be halved. 

If 8^?/" is not too large, then a good approximation to 8^yQ is {hx^y^, 
so that to this approximation, for a function satisfying equation (7.1), 

yi = 2yo-i/-i+(S^)yK.2/o)- 

Thus for such a function we always have a good approximation to the 
function one interval ahead of where we know y". It is this feature which 
makes the numerical integration of such an equation such a straight¬ 
forward process. 

The procedure for two simultaneous equations 

y’’-\-I{^> 2) = 0, y, z) --= o 

or more, with all first derivatives absent, is similar. 

7.21. Change of the interval of integration 

It will not always be advisable to keep the safne interval length 
throughout an integration. It may happen that the suitable interval 
length 8a; varies by a factor 10 or even 100 over the range of x to be 
covered; then the use, over the whole range, of the small interval neces¬ 
sary over part of it might make the calculation so long as to be almost 
impracticable. 
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As already emphasized in § 6.41, it is advisable at any change of 
interval length to take an overlap between the integrations carried out 
with the two different interval lengths. The most usual changes of 
interval length are by factors 2, 2|, or To increase the interval length 
by a factor 2, all that is necessary is to take alternate values of y and the 
corresponding values of y", for two or three intervals before the point 
at which the change of interval length is to be made, difference them, 
and check the formula (7,2) for the function values and differences at 
this new interval, and continue as if these were intervals of the integra¬ 
tion at the new interval. It may be necessary to keep an extra decimal 
in y" to get the same accuracy in y. 

To decrease the interval length by a factor 2, some interpolation is 
required, but only of the simplest kind, namely the ‘half-way’ inter¬ 
polation considered in § 5.21. Suppose that integration has been carried 
out with intervals 8a; — ^ up to a; = X, and it is required to continue 
with intervals 8a; = \h. The integration with intervals Sx — h should 
be carried for one or two intervals beyond a; = X, to give the central 
differences needed in the half-way interpolation. Then y{X — ^h) and 
y"{X — ^h) should be interpolated and y"{X — ^h) also calculated from 
the value of y{X — ^h) and the differential equation, to check. Then 

= v(X-h)-2y(X-ih)+!/(X) 

should be calculated and compared with the value obtained from 
formula (7.2) at the smaller interval. This checks the interpolation of 
y{X—\h)’, this check is most important since the whole subsequent 
integration would be vitiated by a mistake in this value. The integration 
then proceeds, starting from the values of y{X—\h) and y{X). 

Example'. To continue the integration of y" = (1—from x = 0-5, using 
intervals (hx) — 0-05. 

The value of h*y{0'5) for (Bx) = 0*1 will be about — 297 , so the interpolated 
value of y(0'4:5) is 

2/(0-45) = 0-410233-l-i(109262)-TV(3427-l-387i)-l-^{-22,-297) 

= 0-410233-1-5463i-45,3-031 = 0-464402. 

The interpolated value of j/"{0-45) is 

2/"(0-45) = 0-3446-1-^(450)-•j^{-226-297) 

= 0-3446-1-226-1-33 = 0-3704 

(the fourth-difference contribution is negligible), whereas the value calculated 
from the differential equation is 

j/"'(0-45) = [1-{0-45)*](0-464403) = 0-37036, 
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which, to four decimals, agrees with the interpolated value. Thus, starting from 
X = 0-4, we have the following values 


X 

y" 


sv 

y 

0-4 

•3446 

258 


•41023, 

•45 

•3704 

192 

-66 -3698, 

•46440, 

•5 

•3896 



•61949, 


S*i/ 

6416, 

92, 

6509, 


The value of S*2/(0'45) derived from the values of y" is 92^,; the difference between 
this value and the value 924 derived from the values of y is within the tolerance 
for the effect of the rounding error in the interpolated value. The integration can 
therefore be continued from these values. 

Note: In the integration of this equation to this accuracy, it would not actually 
be necessary, or advisable, to decrease the interval of integration at this point; 
this case is only considered here as an example of the procedure. 


The treatment of a change of interval length by a factor 2^ is similar. 
Suppose, for example, the integration has been taken to a: = 0-30 by 
intervals of 0-02 and it is desired to change to intervals of 0-05. The 
values of y and y" at x — 0-25 are obtained by half-way interpolation 
between the values at x = 0-24 and 0-26, y''(0-25) is checked as above, 
and y{0'25) checked by verifying formula (7.2) at the larger interval. 


7.22. Variants of the method 

There are several variants of this method, some of which can be 
combined. 

Instead of evaluating y" for each value of x, forming {y"-\-^^^y'') and 
multiplying this by (Sx)^, we could evaluate {8x)^y" and form 8^y as 

82yo = {8x)Yo-\-mi^^)YV 

At a change of interval length, the entries in the column of (8xYy" 
would be different for the two lengths of interval. 

Another variant is as follows. If we operate on both sides of formula 
(7.2) with the repeated central sum operator = 8~^, it becomes 

Vo = 

If this formula is used, the aggregate contributions from the^S^yo— 
terms in (7.2) are evaluated separately for each interval, instead of 
being built up from contributions from successive intervals. This avoids 
accumulation of rounding and truncation errors in these contributions. 

The process of starting the integration is rather more complicated, 
as initial values for the double sum a^y" have to be evaluated. The 
process does not avoid estimation, since if the integration has been 
carried to x = Xq, then for the end of the step from Xq to Xj we have 

y, = {8x)\o^ylJrhyl-^,,^Yi]\ 
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neither y-^ nor y'l is known at this stage, only the value of a^yi and the 
relation y^ = f{Xi,yi) between y^ and y'l‘, so this is an implicit equation 
for y^y and unless it happens to be linear the solution of it may well be 
more trouble than the integration process. 

This variant requires special treatment at a point at which the interval 
length 8a: changes. 

7.23. Numerov’s method 

If the equation to be solved is Hnear, say 

y’=mt/+gix), (7.6) 

then the solution, to the accuracy given by neglecting the S^y" term in 
(7.2), can be obtained without any estimation as follows. In (7.3) 

= yl-^y'o+y-i, 

and if y" is given by (7.6) this is 
so that (7.3) can be written 

= {8*)*[/oyo+S'o+isSVo]. 

or 82[{l-i(8*)y)s,]„ = (8x)»[/„j,,+9r,+A8VJ. (7.7) 

This treatment is usually ascribed to Numerov.f Written as a relation 
between three successive values of y, (7.7) is 

[l-^s(8x)%]yi = [2+J(Sx)%]!/o-[l-i(S^)y-i]y-i+(8*)'[Sro+A8V»]. 

(7.8) 

A correction for the leading terms in the error of this formula can be 
evaluated by the following method, due to Olver.J Since in this method 
y" is never evaluated, it is convenient to express the corrections in terms 
of the differences of y itself. From formula (4.43) we have 

(8a:)2y" = (7.9) 

so tlrat if y satisfies equation (7,6) 

h-^y = (8a)V(a)!,+j(x)]+ASV-*8'>Ji+sb8"ji+0(8a:)>», (7.10) 

t B. Numerov, PubL de VObserv, astrophysique central de Russie^ 2 (1933), 188; see also 
M. F. Manning and J. Millman, Phys, Rev, 53 (1938), 673, and for a similar method for a 
pair of simultaneous equations, M. V. Wilkes, Proc, Camb. Phil, Soc. 36 (1940), 204, and 
for a system of simultaneous equations with constant coefficients, D. R. Hartree, Journ, 
Inst, Elect. Eng., voL 103, Part B, supplement No. 1 (1956), 82. 

J For a similar treatment of a non-linear equation, see F. W. J. Olver, Proc. Camb. 
Phil. Soc. 46 (1950), 570, § 4. 
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and operation on both sides of this relation with ( 1 +^S^) gives 

S>[{l-i(Sx)y}!/] = (Sx)V(x)!,+ (s+A 8 *?)]-j} 58 *y+,* 5 SV+ 0 ( 8 »)'«. 

(7.11) 

If the terms of sixth and higher orders in Sx are neglected, this is 
equation (7.7). Let z be the solution of this approximate equation, with 
the same initial conditions as those specified for y\ that is 

(7.12) 

and let the solution of (7.11) be 

y = 

Then, on subtracting (7.12) from (7.11), it follows that y satisfies 

S»t{I-,^(8x)y},] = (Sj:)V(a:),- 5 l 58 ««,+n‘fc 8 V+ 0 ( 8 x)'». (7.13) 

From equation (7.13) it follows that t] is of order ( 8 x)^, hence 
differs from 8®2 by terms of order ( 8 x)^®. Hence (7.13) can be replaced, 
with an error of order ( 8 x)^°, by 

8*[{I-i(8x)y),] = (8x)y(x),-A8«z+B‘&i8'z. (’.H) 

Then if z is calculated from formula (7.12) and y from formula (7.14), 
neither of which involve any estimation, the aggregate truncation error 
in y = 2 + rj is of order ( 8 x)®. 


7.3. First-order differential equations 

For a first-order equation the following method, when applicable, 
seems the most convenient. It is based on the use of the integration 
formula 


Vi—Vo = i(8»)[(yi+</;)-i(8x)(8y;'-^8’yJ}], 


(7.15) 


expressing an integral in terms of the integrand and the differences of 
its derivative (see § 6.22). It is appUcable if the function /in the equation 


y' =f{x>y) 

is either given by an analytical formula, so that there is no difficulty 
in evaluating 

dx dy dx dx dy 


to any accuracy required, or if dfjdx and dfjdy can be determined by 
numerical differentiation to the accuracy required in using the formulae 
of the method. 

Two advantages of formula (7.15) have already been pointed out in 
§ 6 . 22 , namely, the small coefficient of the 8 ®y" term in the square bracket, 
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and the fact that if this term (and higher terms) are neglected, so that 
the formula becomes 

Vi—yo = i(s^)[(yo+y'i)—2/0)]. 
it does not involve the values of any quantities at points outside the 
interval through which the integration is being taken. 

Further, the values of y" calculated for use in the integration formula 
can be used to give approximations to the successive values of y by use 
of formula (7.3), namely 

with the term in either omitted or estimated. 

If the numbers occurring in the integration are arranged with those 
referring to the same value of x in a column^ instead of in a row, the 
work can be arranged as in the example on p. 145. 

Example : 

y' = \ — 2xy\ 2 / = 0 at a: = 0 (compare §6.43, equation (6.35)). 

In this case y^' — — 27yy' — 2y. For this equation, it would be possible to use a series 
to start the integration, but here this will not be used, so as to show the procedure 
when the use of a series is not convenient. 

Since in this equation the values of 2y occur in the formula for y' and y'\ it is 
convenient to accumulate values of 2y rather than y. 

For the first interval the values of y^y y^y and yj are available and a first estimate 
of the value of 2y(0-l) is obtained by using these values in the first three terms of 
a Taylor series; this gives 2y(0‘l) = 0-20. From this, approximate values of y'(0*l) 
and y"(0’l) are found, and integration carried through the interval x = 0 to 0*1, 
giving a better value of 2y(0-l), namely 0*1987, from which better values of y' and 
y" at the end of the first interval and a better final value of y are obtained. Use 
of this better value of y does not change the values of y' and y" to the accuracy to 
which they are being used in the integration, and the revised integration through 
the first interval can then be taken-as the first step in the main integration. 

Suppose now that the integration has reached x = 0*3, so that the quantities 
to the left of the heavy line are known. The run of the second differences of S^y^ 
suggests that the value at 0*3 will be about 80, so that 

82[2y(0*3)] = (0*01)2[-1*063 + 0*007] = -0*02112. 

This is written in on the last line but two, and the approximate value of 
2y(0*4) = 0*71989 is built up from it. The values y'(0*4), y'^(0*4) are calculated from 
this value of 2y, and then the integration carried out, giving 2y(0*4) = 0*71987. 
The difference of this value from the trial value 0*71989 is not such as to affect y' or 
y" to the accuracy to which £hey are kept. The integration has now been taken to 
X — 0*4, and the sequence of operations can be repeated for the next interval. 
Notes: (i) Since with the interval taken the values of 8y" are divided by 60 before 
being added to those of y', it is adequate to keep y" to one decimal fewer than y'. 

(ii) It would be practicable to keep another decimal in 2y without using a 
smaller interval 8x. 
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7.31. Another method for a first-order equation 

If in the equation in the form y' = f{x, y), the function / is not such 
that y" can be obtained to an accuracy adequate for use in the method 
of the preceding section, a formula involving only y' and its differences 
can be used. The appropriate central-difference formula, with an error 
term of the same order as that of (7.16), is 

vx-vo = mWo+y'x-hWo+^^ydl ( 7 . 17 ) 

But this is not so convenient, because involves y^, which is not 
known until the next interval, from to y^, has been completed, and 
because the error term is larger than that of formula (7.16). 

When the integration has reached x^^, only 2/o> 2/o> backward 
differences from y^ are available. There is an integration formula using 
backward differences from the beginning of the interval, but it is not 
satisfactory for practical use since the coefficients of the neglected higher- 
order differences are so large; that of the fourth difference is instead 
of ^ for formula (7.16). Even if the trapezoidal approximation is used, 
it is necessary to estimate y^ to give y^, and to adjust this estimate tmtil 
it agrees with the result of integration with the corresponding value of 
yi; and without the value of y^ it is more difficult to obtain a good 
approximation for the first estimate. Even when y^ is known, also 
has to be estimated, and this estimate cannot be confirmed imtil the 
next interval has been completed. An examination of the differences of 
y' and of y in the worked example of the previous section will show the 
advantage of having values of y" available so that the only estimate that 
has to be made is one of h^y". 

7.32. First-order linear equations 

With a first-order linear equation 

y'+My = ff(x) (7.18) 

there is a strong temptation to follow the standard method of textbooks 
on differential equations and reduce the solution of the equation to a 
quadrature by use of an integrating factor. There are occasions for which 
this is useful (see § 7.61 for an example). But the writer’s experience 
has been that in the great majority of cases arising for solution in practice, 
this is a temptation to be resisted, and that it is considerably easier to 
evaluate the solution by numerical integration of the equation as it 
stands than to evaluate the integrals in the solution in quadratures. An 
example has been given in § 1.1. 
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The reason for this can be illustrated by an example. Consider the equation 

y'+2xy = g{x) (7.19) 

with the condition 7/ = 0 at x = 0, and with g(x) a positive function whose maximum 
is of the order of unity, and which is given to six decimals, and is appreciable to 
this accuracy for a; < 20 and negligible for higher values of x [g{x) = for 

example]. The solution y has a maximum of order of magnitude unity, and tends 
to zero as X 00 , The integrating factor of equation (7,19) is e®*, and over the 
range of x where ^(x) is appreciable, e^* increases by a factor of the order of 10^’®. 

The evaluation of the integral J e^'g{x) dx, when the numerical magnitude of 

0 

the integrating factor covers such a wide range, offers numerical difficulties (this 
integral might not even converge for large x, but the six-decimal accuracy to which 
g(x) is supposed given would provide no data for evaluating it beyond x = 20); 
and even in the middle of this range, in the neighbourhood of x = 10, the solution 
would be calculated as the product of two numbers of the order of 10^® and 10~^®. 
These difficulties do not mean that the calculation could not be carried out in this 
way, but they do strongly suggest that it should not be carried out in this way in 
practice. 


Fox and Goodwinf have given a process for numerical integration of 
equation (7.18) in which the linear character of the equation is used to 
avoid the estimation of y mentioned in § 7.31, in much the same way that 
Numerov’s method for a second-order linear equation (7.6) uses the 
linear character of the equation to avoid estimation of S^y". 

For integration through one interval, the trapezium rule with cor- 
rection is ?/i—?/o = \{^x){yQ-^y\)+C. 


Substitution for y' from the differential equation (7.18) and rearrange 
ment of terms gives 


[l-l-|(8a;)/i]yi = [1 —i(8x)/o]yo+i(Sa:)(^o+f^i)+^- ('^•20) 


If 2 is the solution of the relation obtained from (7.20) by omitting the 
correction C to the trapezium rule, that is 

[l-hU^x)f,]z, = [l-i{^x)f,]z,+U^x)(go+gi), (7.21) 

and y = (7.22) 

then = [l-l(S^)/o]^o+C'- (7.23) 

Fox and Goodwin suggest first solving (7.21), and then solving (7.23) by 
an iterative procedure, the correction C being obtained from the values 
of y given by (7.21), (7.22) with the values of y obtained from the 
previous iteration, starting with y = z. 

Another procedure is to integrate from y^ to y^ through two intervals 
8x ^ h and through one interval 8x = 2h, by means of the trapezium- 


L. Fox and E. T. Goodwin, Proc. Camb, Phil. Soc. 45 (1949), 373. 
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rule formula (7.21), eliminate the leading term in the truncation error 
at by Richardson’s ‘^^-extrapolation’ process (see § 7.51; equation 
(7.36)), and continue the integration from the corrected value so obtained. 
If, starting from Zq = = Xq, the results aXx = x^, of using formula 

(7.21) with two intervals 8x = h and with one interval 8a: = 2h are 
and 22 respectively, then 

Vi = (7.24) 

and = Zi-\-\{z^—zl)+0{hxY. (7.25) 

The errors 0(8a:)® in formula (7.24) accumulate over a set of successive 
pairs of intervals, so that the aggregate error over a given range X oi x 
is 0(8x)^; the errors of formula (7.25) do not accumulate, so that the lower 
order of the error in this formula can be tolerated. 

7.33. Second-order equation with the first derivative present 

The most convenient practical treatment of a second-order equation 
with the first derivative present depends on the form of the equation. 
For a linear equation 

= h(x) (7.26) 

the term in y' can be eliminated by the use of 

Y = yex^h j f{x) dxV, 

this gives 

Y''+[ 9 ix)-if'ix)-l{fix)Y]Y = /i(a:)exp[| J /(x) da:] (7.27) 

which reduces the equation to the form treated in § 7.2. This is hkely 
to be a convenient reduction for the homogeneous equation, in which 
h{x) = 0. If ^(a:) is not zero, the exponential factor may make the right- 
hand side of (7.27) vary too rapidly to be convenient for numerical work; 
though since Y has to be divided by a corresponding exponential factor 
to give the solution y required, it may be possible to drop the less signifi¬ 
cant digits of Y as the solution proceeds. 

If the equation is linear in y', though not in y'; 

y''-\-f{x)y'+g{x,y) = 0, (7.28) 

the term in y' can be eliminated by the same change of variable, though 
the resulting equation is not so convenient as (7.27). If, however, g{x, y) 
is periodic in x and a periodic solution of y is required, this reduction 
of the equation is not very convenient, as the function Y will not in 
general be periodic. Then it is probably best to use the equation in the 
form (7.28). 
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A general method of treating the general second-order equation 

^ (7.29) 

is to regard it as two simultaneous first-order equations 

y' = z, z'-]-f{x,y,z) = 0, 

the latter being integrated first in each interval. That is, y" is first 
integrated to give y', and then y' is integrated to give y. The value of y 
at the end of the interval can be estimated by use of 

of which only the term has to be estimated, and for the integra¬ 

tion of y' to give y, y\ is known so that the integration formula (7.16) can 
be used for this integration. If the function/(x, y, y') in equation (7.29) 
is such that y'" can be evaluated to adequate accuracy from values of 
X, y, and y', then the method of § 7.3 can also be used for the integration 
of y" to give y'. For example, for the van der Pol equation 

y"—(1—= 0, 

we have y"'—{\—y^)y’'-\-2yl,y'Y-\-ky' = 0, 

and can use the method of § 7.3 twice in each interval, once to integrate 
y" to give y' and then to integrate y' to give y. This reduction of a second- 
order equation to two first-order equations should not generally be used 
for a second-order equation with the first derivative absent. 

7.34. Equations of order higher than the second 

If it is required to treat numerically an equation of order higher than 
the second, it is best to break down the integration through each interval 
into a sequence of single and twofold integrations. In each interval the 
highest derivative should be integrated first, and the lower-order 
derivatives in succession; then, apart perhaps from the integration of 
the highest-order derivatives, formula (7.16) can be used for any single 
integration required. 

7.4. Taylor series methodf 

There is another method, which is in principle appUcable to equations, 
of suitable form, of any order. Its limitation is that it is only suitable 
for equations in which the relation between the derivatives is given by 
an analytical formula, so that it can be differentiated formally as many 

I See, for example, J. C. P. Miller, British Association Mathematical Tables^ Part- 
volume B, The Airy Integral (1946), Introduction, § 5. 
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times as is required. For example, in the case of the equation 

y’ = x^-y^ (7.30) 

we have in succession 

y" = 2{x-yy'), y'" = 2[l-{t/t/''+(t/')2}] 

= — 2[?/y"'+3?/V''], y"' = — 2[?/y‘'^+4yy"+3(?/'')2] etc. 

It would be possible here to substitute for y' from (7.30) in the first of 
equations (7.31) before differentiating, but this would lead to more 
complicated formulae, and it is better to carry out the substitution 
numerically rather than algebraically. 

Consider first a first-order equation such as (7.30), and suppose that 
the solution has been taken to x = x^, so that y^ is known. Then yl, y'^, 
y^Q,... can be calculated in turn from a set of relations such as (7.31), and 
then y^ can be calculated from the Taylor series 

Vi = yK+8*) = (8a:)V;+^, (S*)y;+.... 


I (7.31) 


It is convenient to arrange the numerical work so that the terms con¬ 
taining odd powers of 8a: and those containing even powers are added up 
separately: 


Then 

and 


^even — 2/o + ^ + ^ (S^)'*2/oT + ^ (S^)Vo*+"** 

Vl ^even“f~^od(l 1 
V—i. ~ ^even ^odd ' 


(7.32) 


(7.33) 


This calculation of y_i, the starting-point for the previous interval, 
from y and its derivatives at a: = a:^ is a very good check; y^ has been 
calculated from y and its derivatives at x_i, so that almost all the numbers 
involved in the calculation of y_i from y^ by (7.33) are different from 
those involved in the original calculation of y^ from y.j. 

There is no particular reason for working with the derivatives them¬ 
selves rather than with convenient multiples of them. In this case the 
convenient multiples are the quantities defined by 


y(n) _ -L 


these are sometimes called ‘reduced derivatives’. That is. 


7(0) _ y, 7(1) _ ^8x)y', 7<2) = i{8x)Y, - ■ 
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Then, for example, (7.30), (7.31) become 

= (Sx)[a:2-{7(0)}2], 

7(2) = (Sx)[a:Sa:-7(‘»7(i)], 

7(3) = ^(Sa:)[(Sa:)2-27(‘»7(2)-{7(i)}2], 

7(4) = -|(Sa:)[7(‘»7(3)+7(i)7(2)] etc., 
and (7.32) becomes 

=n‘”+n'“>+n*>+n‘’+- 

SoM = n"+n=>+n«+- 

No special procedure is necessary for starting the integration. 

For a second-order equation y[ has to be calculated from a Taylor 
series as well as y^. We have 

y'±i = 2 /o±(Sa;)yo + ^(Saj)2y"'±^(8a:)3y|,’^+..., 

and hence 

r«', = (8 j:)s,'±, = 7«>±2rw+3n»±4n*’+--. 

so that if we write 

■SU.„ = 23T+47<«+6n«+..., = r»>+3r«)+6r<«+..., 

the reduced first derivative at x = is 

n'> = ■?;™.+s;aa 

and the check on the integration is provided by 

y(l) O' O' 

~1 ^odd ^even* 

By taking the series (7.34) to several terms, it is practicable to make 
the truncation error of considerably higher order in {8z) than it is in the 
case of formula (7.16) or (7.17), and so to work with a larger interval 8x 
or alternatively to a greater number of significant figures. Results to a 
large number of figures will probably not be required except for equations 
which do satisfy the conditions for this method to be practicable, and 
in such cases it is a very powerful method. 

7.5. Other procedures 

A number of other procedures have been proposed for the numerical 
integration of differential equations. A few will be summarized in the 
following sections. 

7.51. Richardson’s ‘deferred approach to the limit’ 

In most of the procedures so far explained it has been the purpose to 
make each interval of the integration correct, within the tolerance for 


(7.34) 


162 INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS 

rounding error, before going on to the next. This is done by keeping the 
truncation error in each interval less than the rounding error. An alter¬ 
native procedure is to carry out a whole integration using a very simple 
integration formula for which the truncation error is greater than the 
rounding error, and only correcting for the truncation error after the 
whole integration is completed. Such a process has been called by 
L. F. Richardson! a ‘deferred approach to the limit’; a process of this 
kind is applicable to quadrature as well as to the numerical integration 
of differential equations. 

If in integrating the first-order equation 

y' 

we use simply the trapezoidal formula 

= yi—yo = U^^)ifo+fi)y ( 7 . 35 ) 

the result y at a given value of x will depend on the interval length 8a; 
used in the integration as well as on x. Let us express this by writing 
this result as y{x,8x)] the solution of the differential equation is the 
limit of this as 8x 0, namely y{x, 0). 

Now in each interval the error in 8y calculated by (7.35) is of order 
(8a;)®. The number of intervals required to cover a given range of x is 
inversely proportional to Bx; hence the aggregate truncation error is of 
order (8a;)®. Such an error in y results in an error in y' of order (8a;)®, 
which makes an additional error of order (8a;)® in each By, which is of the 
same order as the truncation error in that interval alone. Thus the 
aggregate error at any given x is of order (8a;)®. 

If now two separate integrations are carried out, using the same 
integration formula (7.35), with different interval lengths (8a;), then 
the leading term in the aggregate truncation error can be eliminated by 
extrapolating to Bx = 0, linearly in (8a;)®, at each value of x. The most 
convenient way of doing this in practice is by use of one set of intervals 
8a; = h and another set 8a; = 2^; the extrapolation process is represented 
graphically in Fig. 12. A convenient numerical process is represented 
by the formula 

y{x, 0) = y{x, h)—l[y{x, 2h)~y{x, 7i)]. (7.36) 

This process has been called by Richardson ‘A.®-extrapolation’. It is 
important to ensure that cumulative rounding errors do not vitiate this 
extrapolation to 8a; = 0. 

It is in principle possible to carry out this process of extrapolation to 
f L. F. Richardaon, Phil, Trans, Roy, Soc, 226 (1927), 300, 
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8a: = 0 from results calculated for more than two different interval 
lengths, but this is not a satisfactory procedure in many cases. 

If the aggregate error over a given range of x is 0{8x)*, as with 
Numerov’s method (§ 7.23), a similar process of extrapolation to 8a: = 0, 
linearly in (8a:)'*, can be used. 



7.52. Iterative processes 
The solution of the equation 

y'=fi^,y), y{xo) = yo 

can formally be obtained by constructing a sequence of functions y(n){x) 
y(n+M = yo+ J 2/(n)(^)} dx. (7.37) 


Xo 


If the integral is evaluated by quadrature, this process of iterative 
quadrature is the numerical equivalent of Picard’s process in the formal 
analytical theory of differential equations. It is sometimes useful for 
starting an integration, but unless a good approximation is available to 
use as a first approximation y^ix) in the right-hand side of (7.37), it is 
seldom useful for carrying the integration of an equation with one-point 
boundary conditions over a range of x, because the convergence of the 
successive functions y(„) to the solution of the equation is too slow. A 
form of iterative quadrature may, however, be useful in dealing with 
problems with two-point boundary conditions (see § 7.61). 

As an example of another, more practical, kind of iterative process, 
consider the equation ^ 

A sequence of functions y(„) can be formed by 

Here only the leading term on the right-hand side depends on the 
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function currently being evaluated; the ‘correcting’ terms, involving 
h'^y" are derived from the previously calculated function y(n-i)- This 
avoids any estimation of and enables 8^" and 8®y", and higher 
differences, to be included. 

A different iterative process is obtained if y" on the left-hand side of 
(7.38) is expressed in terms of y and its differences. Substitution for y" 
in formula (7.9) gives 

Then a sequence of functions y(„) can be formed by means of the iterative 
formula 

Vin+a = iSxf[aY{x, y(n+j}]+h^^yU)—^^^y(n)]-\-0{Sx)\ (7.41) 
where is the twofold sum operator as in § 7.22. Here the ‘correcting’ 
terms in the evaluation of y^n+j) ^^e expressed in terms of the differences 
of the previous function y^^) itself, instead of in terms of the differences 
of its second derivative. A similar treatment can be applied to first-order 
equations, and various examples of its application have been given by 
Fox and Goodwin.f 

7.53. The Madelung transformation 

For evaluating oscillatory solutions of a second-order homogeneous 
linear equation with the first derivative absent: 

y"+/(^)y = 0, (7.42) 

a transformation due to MadelungJ is sometimes useful. 

Let us write y = F(a:)expj^i J g{x)do^, (7.43) 

where F and g are to be real; this is equivalent to describing the oscil¬ 
lating function y at each point in terms of a local amplitude F{x) and 
local phase <j){x) = j g{x) dx. The real and imaginary parts of (7.43) 

clearly give two linearly independent solutions of (7.42). Substitution 

of (7.43) into (7.42) and separation of real and imaginary parts gives 

F"—= 0, (7.44) 

2F'g+Fg' = 0. (7.45) 

The second of these two equations is integrable and gives 

F^g — const. = A (say), (7.46) 

and substitution in (7.42) then gives 

= 0 . 

f L. Fox and E. T. Goodwin, Proc. Camb, Phil, Soc, 45 (1949), 373. 

X E. Metdelung, Zeit.JiXr Phys, 67 (1931), 516, 


(7.47) 


INTEGRATION OF ORDINARY DIFFERENTIAL EQUATIONS 155 

Thus the evaluation of the two functions F and g can be separated, 
equation (7.47) being first solved for F, and g then determined. 

Equation (7.47) is not hnear, but it may be easier to integrate than 
the original equation (7.42), since the rapidly oscillating part of y has 
been taken out by the factor exp[^i J g{x) cixj, and the function F de¬ 
scribing the local amplitude of this oscillation will often vary relatively 
slowly, f 


7.54. The Riccati transformation 

For equation (7.42) with/(x) negative, the Riccati transformation 

V = y'ly = d(\ogy)jdx, ri'-{-ri^-\-f{x) = 0 (7.48) 

is sometimes useful, especially if f{x) is negative over a considerable 
range of x. f{x) is negative and slowly var 3 dng, and |/(a;)| is large, a 
solution of equation (7.48) can sometimes be obtained by writing it in 
the form 




(7.49) 


and solving this by iteration. The first approximation is 

V = [-/(*)]•; 

numerical differentiation then gives an approximation to y' in (7.49) and 
hence a better value of 17 . This is one of the few situations in which 
numerical differentiation may be useful as a tool in a practical numerical 
process. 


7.6. Two-point boundary conditions 

As an example of the treatment of two-point boundary conditions, 
consider the solution of 

subject to the conditions 

y = yo at X = Xo, y = yt, at x = 6 . 

A step-by-step integration, starting from x = x^, has to start from 
definite numerical values of y at the beginning and end of the first interval, 
and the result at any later value of x is determined by these two values 
of y. The former is given to be y^^, but the latter, y^ = y(X(,-f Sx), is not 
specified. If the variation of the solution y{h) at x = 6 with the value 
of is not too rapid, the following process can be used to find the solution 
satisfying the condition x = h. A set of integrations is carried out for 
a set of trial values of y^, and the value of y{h) obtained as a function of 
y^. Interpolation (graphical or numerical) can then be used to obtain a 

t For an example of the use of this transformation, see D, R. Hartree, R. L. Kronig, 
and H. Pedersen, Fhyaica^ 1 (1934), 895. 
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close approximation, say to the value of which gives a solution for 
which y{b) has the required value A further one or two integrations 
with values of in the immediate neighbourhood of then enables the 
solution satisfying the given condition at a; = 6 to be obtained by linear 
interpolation. 

If the value of y{b) is very sensitive to the value of y^ such a process 
is not practicable in this simple form. Consider the variation of y at a 
fixed X with the trial value of this is dy{x)ldyy. This is a function of x, 
and, for a non-Unear differential equation, corresponds to a ‘comple¬ 
mentary function’ of a hnear differential equation. It might behave 
approximately as in which case it would increase by a factor of about 
10^® over a range of x from 0 to 5. This would mean that unless the 
choice of y^ were correct to 0*00001, all trial solutions at x = & = 5 
would have values of |y(6) | of the order of 10® or larger. If the condition 
at X = 6 were y{b) = 1, interpolation between two trial solutions with 
values of y^ differing by 0*00001 would determine y^ closely but would 
not determine the solution at all weU except near x = 0. Further, the 
effects of rounding errors in the early intervals of the integration build 
up in much the same way as the function dy{x)/dyi so that it would be 
necessary to keep a large number of guarding figures, and probably to 
work to 15 or 20 decimals. 

Such a situation is not rare; in the writer’s experience, it is more hkely 
than not to occur in equations with two-point boundary conditions which 
arise in real problems (as distinct from those which are made up to serve 
as textbook examples). In such a situation, however, a procedure of the 
same kind can be used, proceeding by stages in the x direction. Two 
solutions with different values of y^ are carried to such a value of x, say 
x^, that their behaviour indicates clearly enough whether the required 
solution lies between them or not; if not, other solutions are evaluated 
until a pair is found between which the required solution does lie. Let 
these be yj and yu, with values (yi)i and (yii)i at the end of the first 
interval. 

From the behaviour of the solutions yj and yu at x = x^ and the 
expected behaviour of the solution required, an estimate is made of the 
fraction p of the difference between the ftmctions yj and yu such that 
yj-j-p(yu—yi) is a fair approximation to the required solution. Linear 
interpolation will probably not be valid at x = x^j but should be good 
enough to give one decimal in p, which is all that is wanted. Another 
solution, yjjj, is then started, not from x = x„ but from some smaller 
value Xj of x at which linear interpolation between the solutions yj and 
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yii is valid to the accuracy to which the calculation is carried. Whether 
linear interpolation is vahd can usually be tested by comparing (a) the 
value of y"{x^ interpolated hnearly between the values for solutions yi 


and yii, that is 




and (6) the value of y"{x^ calculated from the interpolated value of y{x^. 
If the difference between these two values oiy"{x^ is not enough to affect 
the last digit of y(Xj-}-Sx), then, in the usual contexts in which this pro¬ 
cedure is required, the hnear interpolation is adequate. 

Depending on the behaviour of ym, either another solution is started 
from Xj, or a solution is started by linear interpolation between y^n and 
«/ii or yi at a point farther out, by a repetition of the process for 
selecting and starting the evaluation of the solution yjjj. This process 
may have to be repeated several times before the value a: = 6 is reached. 


7.61. Iterative quadrature 

In the process considered in the previous section the solution satisfying 
the two-point boundary conditions is reached by evaluating a sequence 
of functions each of which does satisfy the differential equation but does 
not satisfy all the boundary conditions. An alternative procedure in 
some cases is to approach the solution required through a sequence of 
functions each of which satisfies all the boundary conditions, but does 
not satisfy the equation. 

Consider, for example, the equation 

(7.50) 

with boundary conditions 

y =z 0, y'= 0 at X = 0, y'->■ 1 as xoo. (7.51) 
Let y(n){x) be a sequence of functions defined by 

!'rn+i>= -(l+3'5.>Wn+i>- (7-52) 

If at any stage of the work y(j^) is a known function of x, this is an equation 
for the next function of the sequence, namely y(n+i)'> if is linear and 
homogeneous in this unknown function and there is no difficulty in 
obtaining a solution of (7.52) satisfying all three of the boundary con¬ 
ditions (7.51). One integration gives 

y'U+i) = ^expj^- J (1+yfn)) dx^> 
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where A is, so far, an undetermined integration constant. Another 
integration gives 



the condition ?/'(0) = 0 has been satisfied by choosing the lower limit of 
the integral; the condition ^'(oo) — 1 can now be satisfied by choice of 
A , and gives 



and another integration from lower hmit x = 0 satisfies the condition 
on y(0), giving 



2/(n+l) = 



This may appear a rather elaborate form of equation (7.50); however it 
contains the boundary conditions (7.51) in addition, and is in fact 
quite convenient for numerical work. 

Unless ^(n+i) = y(n) to the accuracy of the numerical work, is 
not a solution of equation (7.50), so that the separate members of the 
sequence y(n){^) are not solutions of the equation though they do satisfy 
all the boundary conditions. But if the process converges, in a numerical 
sense that after a finite number of repetitions of the iterative process 
^(n+i) becomes equal to y(„) to the accuracy to which the numerical work 
is taken, then to this accuracy such a function y(„+i) is a solution of the 
equation (7.50). 

A process of this kind, when available, is particularly useful in cases 
in which, using a step-by-step integration, y is very sensitive to y-^. 
This sensitiveness is an indication of a kind of instability in the step-by- 
step process; but this instability does not correspond to any instability 
in the physical system to which the equation refers, just because in the 
physical system the behaviour of the system is fixed at both ends of the 
range. It does not appear either in a treatment such as that of the present 
section, which approaches the physical situation more closely in that it 
insists at each stage that the approximate solution is tied at both ends, 
however it behaves intermediately. 

Another method, which also works through a sequence of approximate 
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solutions satisfying the boundary conditions at both ends of the range, 
is an application of the ‘relaxation’ process of Southwell (see § 8.5 and 
end of § 8.6); and another process is considered in § 8.6.f 

7.62. Linear equations with two-point boundary conditions 

For Linear differential equations with two-point boundary conditions 
there are special methods which are not applicable to non-linear equa¬ 
tions. The equations and boundary conditions together can be divided 
into three main classes: 

(i) Homogeneous equations with conditions y = 0 (or more generally 
homogeneous boundary conditions) at both ends of the range, and the 
further condition that the solution should not be identically zero. 

(ii) Homogeneous equations with the condition y given and non-zero 
(or more generally an inhomogeneous boundary condition) at one or both 
ends of the range. 

(iii) Inhomogeneous equations. 

The first of these classes will be considered in the next section; the 
present section is concerned with the second and third, the treatments 
of which are similar. 

Consider first the inhomogeneous equation 

y"-\-f{^)y = (^-63) 

with y{a) and y{b) given; either or both may be zero without affecting 
the argument. If Y is any solution of equation (7.53) satisfying the 
condition Y = y{a) aX x = a, and z is a solution of the corresponding 
homogeneous equation 

z"-\-f{x)z = 0 (7.54) 

satisfying the condition z = 0 at a; = a, then, for any constant a, 

y = 7-f az (7.55) 

is the solution of (7.53) satisfying y = y{a) aX x = a. In principle, the 
required solution can be determined by evaluating Y and z by integration 
of the respective equations and then forming the linear combination 
(7.55) so as to make y = y{h) aX x = h\ and this may be a practicable 
procedure. However, if over the range x = a to x = b the solution 7 
becomes large compared with the solution y to be determined, this 
procedure may result in the unsatisfactory situation that y is evaluated 
as the small difference of two relatively large numbers; for example 7 

t For a much fuller treatment of numerical methods for differential equations with two- 
point boundary conditions see L. Fox, Thz Numerical Solution of Two-point Boundary 
Problems in Ordinciry Differential Equations (Clarendon Press, 1967). 
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and az might be of the order of 10^ and y of the order 10“^, and then 
eight correct significant figures would be needed in Y and z to give one 
figure in y. Such a situation will usually occm: if 6 = oo and the comple¬ 
mentary function increases without limit as a: oo, for example if it 
behaves asymptotically like a:*' (A: > 0) or e*'® {k > 0). 

In such cases it may be practicable to obtain a solution satisfying the 
required boundary conditions by integrating outwards from x = a and 
inwards from x = h, and matching the solutions at two intermediate 
values of x, say x = and a; = (or perhaps at one inter¬ 

mediate value). 

Let be any solution of (7.53) satisfying the condition Y = y{a) 
at a: = a, and a solution of (7.54) with z{a) = 0; then, as before, 
^ut+“^out ^ solution of (7.53) satisfying the condition at a; = a. 
Also let I^ln be any solution of (7.53) satisfying the condition Y = y{b) 
a,t X = b, and Zj^ a solution of (7.54) with z(6) = 0, then 
solution of (7.53) satisfying the condition at a; = 6. The outward inte¬ 
gration is taken to a; = Ag and the inward integration to a; = X^, so that 
they overlap over the range X^ ^ x ^ Xg. To match the results of the 
inward and outward integrations we require 

~ ^n(-^i)d*^^in(-^i) (7.56) 

and Y^^^{X 2 )+<xz^^^{X^) = Y^^{X^)■Y^z^^(X^). (7.57) 

These equations can be solved for a and and the solution y constructed 
from these values, but it may be more convenient to proceed as follows. 
From (7.56) and (7.57) 

^U t(-^2)~^ D(^2)+«2out(-3L2) _ 

which can be solved for a. Then for x ^ X 2 , y can be evaluated from 

y{x) = 5^out(«)+“2:out(«). (7.68) 

and in particular 

Also for a; > X^, y{x)—Y^^{x) = ^z^^{x), so 

y{X,)-Y,j,x~)-z,j,x,y 

from which y can be constructed for a; > X^. The agreement between 
the values of y, for X^ < a; < X 2 , calculated from (7.58) and from 
(7.59) provides a good check that the matching of the results of the 
inward and outward integration has been carried out correctly. 
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An alternative matching procedure is to match both the values of y 
and of y' for the inward and outward integrations at a single inter¬ 
mediate value of X, say x = Then equation (7.57) is replaced by 

and the subsequent argument is similar. This procedure has two prac¬ 
tical disadvantages compared with the above procedure for matching y 
only, but at two values of x, namely it requires a process of numerical 
differentiation (§ 6.7) to give the values of the derivatives, and it does 
not give the check provided by the overlap region Xj < x < Xg when 
the solutions are matched at two values of x. However, it may be useful 
if the Riccati transformation is used for the calculation of comple¬ 
mentary function z, since no differentiation is then required to obtain 
z', and an alternative check of the matching procedure could be de¬ 
vised. 

For a homogeneous equation with y — y(a) 7 ^ 0 at x = a, or 

y = y(b) 7^ 0 at X = 6, 

or both, the procedure is very similar, being now a solution of 
F'-f/(x)r = 0 with y = y{a) at x = a, andFjj, a solution with y = y{p) 
at X = 6; if one or other of y(a) and 7/(6) is zero, the corresponding 
function Y is omitted from the argument. 

An alternative procedure, based on a matrix treatment of the finite- 
difference form of the differential equation and boundary conditions, is 
considered in § 8 . 6 . 

7.63. Factorization method 

For the equation y"—k^y = g{x) ( 7 . 60 ) 

with ^(x) ^ 0 as X ^ 00 and boundary conditions y ^ 0 as x ^ 00 and 
any linear boundary condition at x = a, a process based on the factori¬ 
zation of the operator on the left-hand side may be convenient. This 


factorization gives 

(-■ 

\dx 



so that if we write 



(7.61) 

then 


II 

1 

(7.62) 


From equation (7.61) it follows that u -> 0 asx ^ co, so equation (7.62) can 
be integrated inwards from a known condition for large x, and this inte¬ 
gration is stable since the complementary function is e+*-^ and decreases 

M 


«013 
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as X decreases. This inward integration can be started from the greatest 
value of X at which g{x) ceases to be negligible to the accuracy required 
in the calculation. Its value at a: = a gives t>{a) = y'{a)-\-ky{a) for the 
solution satisfying the boundary condition at infinity, and from this value 
of v{a) and the boundary condition at a; = a, the value of y{a) can be 
derived, and forms the boundary condition for the outward integration 
of equation (7.61); this integration is also stable, since the comple¬ 
mentary function is now e~**, and again decreases in the direction in 
which the integration is being taken. 

This process of treating a second-order equation with two-point 
boundary conditions by carrying out two integrations, each of a first- 
order equation, over the whole range between the values of x, and 
satisfying one boundary condition at each integration, can be extended^ 
to equations with variable coefficients and inhomogeneous boundary 
conditions at both ends of a finite range of x. 

7.64. Characteristic value problems 

In the equation considered in § 7.6, the parameter which is available 
for adjustment in order that the solution should fit the two-point 
boundary conditions was y^, and in § 7.62 it was the multiple of a comple¬ 
mentary function which had to be added to a particular integral. In 
either case, adjustment of the parameter is equivalent to adjustment 
of an initial condition in a step-by-step integration. 

For a homogeneous hnear second-order equation with homogeneous 
boundary conditions, the adjustable parameter is a constant in the 
equation itself, for example the constant A in the equation 

y"-\r[^-\-f{x)\y = 0 (7.63) 

with boundary conditions 

y{a) = y{h) = 0, (7.64) 

and the further condition that the solution should not be identically zero. 
For such an equation and boundary conditions there may be no solution 
(other than y = 0 ) unless A has one of a set of discrete values, which 
may be finite or infinite in number. These values of A are called the 
‘characteristic values’ of the differential equation with these boundary 
conditions, and the corresponding solutions y are called ‘characteristic 
functions’. Solution of the equation involves determination of one or 
more of the characteristic values of A as well as the corresponding 
characteristic functions; in some cases determination of the characteristic 

t See E. C. Ridley, Proc. Gamb. Phil. Soc. 53 (1957), 442. 
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values may be more important than determination of the solutions of 
equation (7.63) themselves. 

The determination of characteristic values and functions can be carried 
out by evaluating trial solutions with trial values of A. One process is 
similar to that considered in § 7.62 for an inhomogeneous equation with 
two-point boundary conditions, by carrying out an outward integration 
from X = a and an inward integration from x = b, and matching them 
at some intermediate tadius, say x = X. The results of these two inte¬ 
grations will be written and y,„. 

Since equation (7.63) and the corresponding boundary conditions are 
homogeneous, it follows that if y is a solution, then so is Ay for any 
constant value of A. Hence the condition for matching the results of 
the outward and inward integrations a,t x = X now consists not of two 
independent relations yi^{X) = y^^tiX) and y;n(^Y) = but the 

single relation [y’(X)/!,(A-)],„ = [</'(A')/j,(A')]„,. 

expressing the property that if the arbitrary multiplying constants in 
the inward and outward integrations are so chosen that yjii(X) = yoyt(X), 
then also yin(-A^) must be equal to yout(^)- each integration is carried 
one or two intervals beyond x = X, then values of y'{X) can be evaluated 
from the central-difference formula (6.54) for numerical differentiation; 
enough figures must be kept in Sy to enable these determinations of y'(X) 
to be carried out to the accuracy required by the rest of the work. Graphs 
of y'{X)/y{X) against A for the outward and inward integrations, plotted 
on the same piece of paper, will assist the choice of successive trial values 
of A. 

The degree of mismatch between the results of inward and outward 
integration for a trial value of A, as measured by the difference between 
the values of y'{X)jy{X) for the two integrations, may be used directly 
to estimate an improved trial value.f Let Ay be the difference between 
two solutions of equation (7.63), both satisfying the condition y(a) = 0, 
with values of A differing by AA, this difference being taken between the 
values of the two solutions at the same value of a;. Then (y+Ay) satisfies 

the equation (j,+Aj,)'+[(A+AA)+/(x)](!i+Ay) = 0. 

SO that Ay satisfies the equation 

Ay"-f [(A+AA)+/(a:)]Ay-f (AA)y = 0 
exactly, or to first order 

Ay'-f [A-f/(a;)]Ay+(AA)y = 0. 

I See E. C. Ridley, Proc. Carnb. Phil, Soc. 51 (1955), 702. 


(7.65) 
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Now consider the result of integrating equations (7.63) and (7.65) 
outwards from conditions y{a) = A?/(a) = 0 . By multiplying equation 
(7.65) by y and (7.63) by —Ay and adding, it follows that 

{y^y"-y’'^y),^, = -mylr^t- ( 7 - 66 ) 

The left-hand side of (7.66) is 

^(yAy'-y'Ay)o„t, 


and yAy'—y'Ay = y^^iy'ly); 

also y{a) = Ay (a) = 0 ; hence integration between limits x = a and 
X = X gives 

My'(X)ly(X)]^ = -(^)[J' <te]/[S'out(JC)]’‘- P-67) 

Similarly for the inward integration of equations (7.63) and (7.65) from 
conditions y{b) — Ay( 6 ) = 0 , 


A[y'(A’)MX)],. = +(AA)[| yl,dx 


[yini^)Y- 


(7.68) 


Now if for a trial value of A the values of 


[!,'(A)/!,(A)],„ and ly'I.X)MX)U, 

do not match, the value of AA required to make them match is the value 
such that 


[y'(A')/!,(A)]„,+A[v'(A)/!/(X)]„, = [!,'(A)/!,(A)],„+A[y'(A)/!,(A)],.. 
and substitution from (7.67) and (7.68) gives, to first order in AA, 


rX 

b 

J 2 /out 

J yfn dx 

a 1 

1 ^ 

IvUiX) ' 

' 2 /fnWJ 


AA = [y'(Z)/y(X)U- 


[y'{X)/y{X)\^. 


(7.69) 


In using this formula, it must be remembered that it is not exact, but 
only first-order in AA; it is most useful for improving an approximation 
to A which is already fairly good. In some cases it may be practicable to 
take X = b in formula (7.69); that is, to dispense with the inward 
integration. 

Another way of matching an inward and an outward integration is to 
match the ratio of the values of y at two values of X, that is to determine 
A so as to make [y(-X^ 2 )/z/('^i)]in = [Z/(-^ 2 )/ 2 /(-^i)]ouf This avoids the need 
for differentiation to give values of y', but there is then no simple formula 
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like (7.69) by means of which the degree of the mismatch with one trial 
value of A can be used directly to estimate a better value of A.f 

For many forms of the function/(a;) for which solutions of (7.63) are 
wanted in practice, there is a finite least value of A; this can often be 
determined approximately by using Rayleigh’s principle.J This states 
that if z is an approximation to the solution of (7.63) for the lowest 
value of A, then j, j, 

J z{z"-\-f{x)z} dxj J z^ dx (7.70) 

a 'a 

b 

differs from A by a quantity of order j {y —z)^ dx, so that a rough estimate 

a 

of z substituted in (7.70) gives a fair value of A. This may often give a 
good first trial value to use in the numerical integration of equation 
(7.63). 

Characteristic value problems may also occur with inhomogeneous 
equations; for example we may require the solution of the equation 

y"+[A+/(a:)]y = g{x) (7.7i) 

which satisfies the boundary conditions y{a) = y{h) = 0 and the ‘nor- 

b 

mahzing’ condition = 1. For any value of A which is not a 

a 

characteristic value of equation (7.63) with the same function f{x) and 
the same boundary conditions,§ there is a solution satisfying the boun¬ 
dary conditions, and it is unique, but it does not in general satisfy the 
normalizing condition. The values of A for which the solution satisfying 
the boundary conditions also satisfies the normalizing condition are the 
characteristic values of equation (7.71) with these conditions. They can 
be found by using the methods of § 7.61 to find the solution of equation 
(7.71) satisfying the boundary conditions, for a series of trial values of A, 

b 

evaluating j y^ dx for each solution, and using each result to estimate 

a 

a better trial value. 

f For another procedure, see § 8.7. See also W. E. Milne, Journ. of Research of Nat. Bur. 
of Standards^ 45 (1950), 245. 

X See G. Temple and W. G. Bickley, Rayleigh's Principle (Oxford, 1933). 

§ If A is such a characteristic value, equation (7.71) has no solution satisfying the 
boundary conditions unless the function g(x) satisfies some special conditions. 


VIII 


SIMULTANEOUS LINEAR ALGEBRAIC 
EQUATIONS AND MATRICES 

8.1. Direct and indirect methods for simultaneous linear 
equations 

The necessary and sufficient condition that a system of linear simul¬ 
taneous equations should have a solution, and that the solution should 
be unique, is that the determinant of the coefficients should be non-zero. 
In this chapter, except in § 8.7, it will be assumed that this condition is 
satisfied, so that we shall only be concerned with the determination of a 
solution when a unique solution exists. 

Any textbook of algebra shows that the solution of such a set of 
equations can be expressed in terms of ratios of determinants, and one 
way of evaluating the solution of the equations would be to evaluate 
these determinants numerically. But though there may be no one best 
way of evaluating the solution, it can be said with some certainty that 
the direct evaluation of the determinants and of the expression for the 
solution in terms of them is never the best way, though of course the 
evaluation of a solution by any other method must come in the end to 
the same thing as the evaluation of the solution in terms of determinants. 
The general form of a set of such equations will be written 

®21^1"i“®22^2"l“®23^3"!“••• ~ ^2> 

or shortly J = 6,- {i = 1, 2 ,..., n), 

j 

or in matrix form Ax = b. 

The equation ^ = i>i with any particular value of i will be called 

the Sth equation’ and n will be used throughout for the number of 
equations and of unknowns. I will be written for the unit matrix, that 
is the matrix of which all the diagonal elements are 1 and all the non¬ 
diagonal elements 0. 

There is probably no one way for evaluating the solution of such a set 
of equations which can be said to be the best in all circumstances. 
The most effective practical method to use depends on various character¬ 
istics of the equations and of the solutions required and on the experience 
of the individual who has the task of carrying out the numerical work. 


( 8 . 1 ) 
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The character of the equations is concerned with such matters as whether 
the coefficients are all small integers or not, whether many are zero, and 
whether those which are non-zero are arranged in some systematic way, 
whether they are all exactly known or are subject to uncertainty as a 
consequence of being either experimental measures or results of other 
calculations subject to rounding errors, and whether the diagonal 
coefficients are large compared with the non-diagonal coefficients or not. 

Relevant characteristics of the solutions required are whether a 
solution is wanted for one set of values of the b/a only or for many such 
sets, and whether the characteristic values of the matrix A are required 
as well as the solution of the equation. 

There are two main kinds of method, sometimes called ‘direct’ and 
‘indirect’. 

‘Direct’ methods are those in which one apphcation of the computing 
procedure leads to the solution, to an accuracy depending on the nominal 
accuracy of the calculations. The evaluation of the expression for the 
solution in terms of determinants, and the method of elimination, are 
examples. 

‘Indirect’ methods are those in which the solution is approached by 
successive approximation, by a number of repetitions of the same com¬ 
puting procedure. For hand calculations an advantage, when such a 
method is appUcable, is that in the early stages only a limited number 
of figures need be kept, and the accuracy can be increased as the solution 
is approached. 

If an approximation X 2 = general x^ = to the 

solution of the equations has been obtained, the quantities 

^i = ( 8 . 2 ) 

} 

are called the ‘residuals’ of the various equations: they measure the 
extent to which the approximate solution x^ = fails to satisfy the 
equations. The corrections = {x ^—to the approximate solution 
satisfy the equations 

= -Ri (8.3) 

} 

with the same coefficients as the original equations. In various methods 
of evaluating a solution it is possible to proceed by a process of approxi¬ 
mation, obtaining first an approximate solution, calculating the residuals 
for this approximate solution, then solving (8.3) for corrections to the 
approximate solution, and perhaps repeating this process. 
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8.11. Matrices 

If, as assumed, the determinant of the coefficients is non-zero then 
the matrix A of the coefficients has an inverse A“^, and the solution is 

X = A-‘b. (8.4) 

If many solutions for different sets of values b^ are wanted, then it may 
be worth while to evaluate A“^ first and then to evaluate solutions 
from ( 8 . 4 ); one process for the inversion of a matrix is considered in 
§ 8.3 and another in § 8.41. In general the calculation of A“^ will be 
affected by rounding errors, so the matrix used as A“i in evaluating a 
solution from (8.4) will not be exactly the inverse of the matrix A of 
the coefficients of the original equations. The results of evaluating 
(8.4) with this approximate A~^ may therefore have to be treated as an 
approximation ^ to the solution and improved as explained at the end 
of the previous section. 

A matrix in which all elements below the diagonal are zero is called 
an ‘upper triangular’ matrix and one in which all the elements above the 
diagonal are zero is called ‘lower triangular’. The determinant of a 
triangular matrix is the product of its diagonal elements. 

8.12. Ill-conditioned equations 

If the determinant D of the coefficients is expanded in terms of the 
elements of any one row or column, for example 

^ — ^ (8-5) 

(the sum being over k only, j being fixed), it may happen that D is small 
compared with some of the individual terms in this sum. Then the 
value of D, and so the solution of the equations, is very sensitive to small 
changes in the values of the coefficients, and an approximate solution 
obtained by a numerical method which is subject to rounding errors is 
hkely to be very sensitive to these errors. If, for example, D is of the 
order of unity but some of the individual terms in ( 8 . 5 ) are of the order 
of 2000 , then a change of 0-1 per cent, in the coefficient in one of these 
large terms may change the value of D from +1 to — 1 , and the solutions 
in the two cases, and for intermediate values of this coefficient a^j., may 
be entirely different. 

A very elementary example is provided by the equations 
x->r2y = 4, 1000x+2001y = 4003 

for which D = 2001 — 2000 = 1 , and the solution is 

a: = -2, y 3. 
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If the coefficient of y in the second of these equations is changed by 
—O’l per cent, to 1999, the solution becomes 

a; = 10, y = —3 

and if it is changed by -(-0*1 per cent, to 2003, the solution becomes 

y = 1. 

If the coefficients in these equations are known exactly, then the solution 
can be determined to any accuracy required. But if the coefficients are 
subject to some uncertainty, either through being derived from observa¬ 
tions which can only be made to a finite degree of accuracy, or through 
being themselves results of other calculations which may be affected 
by rounding errors, then clearly not even the sign and first significant 
figure of the solution can be determined unless the uncertainties in the 
coefficients are less than 0-1 per cent. A set of equations for which D is 
small compared with some of the individual terms in the sum (8.5) is 
called ‘ill-conditioned’. 

It is sometimes said that a set of equations is ill-conditioned if the 
determinant D of the coefficients is small; but this is an inadequate 
statement because the relevant standard of smallness is unspecified. 
Consider a set of thirty equations for which 2> = 1. The relations 
between the variables expressed by the equations are not altered if each 
equation is multiphed by 1000; but D then becomes 10®®, which is not 
‘small’ in any ordinary use of the word. 

‘Ill-conditioned’, applied to a set of equations, is sometimes used 
merely as a quahtative term of abuse; but it is capable of being given a 
quantitative significance. Let A^”^ be the characteristic values 

of the matrix of the coefficients and let |A<”*^|, lA^^^I be the greatest 
and least of the quantities |A^’^|; then lA^-^^/A^”*)] is a quantitative measure 
of ill-conditionedness; when this ratio is nearly unity the equations are 
well-conditioned, when it is large compared with 1 they are ill- 
conditioned; when A^”*) is zero the determinant D of the coefficients is 
zero and either the equations have no solution or the solution is not 
unique. 

Unfortunately the numerical determination of this measure of ill- 
conditionedness is as long a process as the evaluation of the solution of 
the equations, and the same apphes to some other measures of condition 
which have been suggested by Turing, f So it is not very useful in practice 

f A. M. Turing, Quo/ri, J. Mech. and Applied Math. 1 (1948), 287. 
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for giving advance warning that the equations concerned are ill-con¬ 
ditioned. In many cases the intermediate results obtained in the course 
of the numerical process of solving the equations exhibit characteristic 
symptoms when the equations are ill-conditioned. These symptoms 
depend on the particular process and will be mentioned in the course 
of consideration of the individual processes. 

Sometimes inspection of a set of equations will suggest that they are 
ill-conditioned. Expressed geometrically, any one of the equations is 
the equation of a hyper-plane in n-dimensional space, and the coefficients 
in the equation are the components of a vector normal to this hyper¬ 
plane. If these normals are all in much the same direction then the h 3 q)er- 
planes are nearly parallel so that their intersections are at very acute 
angles, and the common point of them all, which represents the solution 
of the equations, is not well determined. Eor example, it is clear on 
inspection that for the equations (constructed by T. S. Wilson and quoted 

by J. Morris)! fe,+ 7 x,+ 6 * 3 + 5 x. = 23 

+ 10 x 2 - 1 - 8 x 3 -!-7= 32 I . 

6 xi-t- 8 x 2 + 10 X 3 - 1 - 9 x 4 = 33 I 

5 X 4 - 1 - 7 X 2 + 9 x 3 - 1 - 10 X 4 = 31 j 

the normals to the hyper-planes make only small angles with each other, 
so that this set of equations is ill-conditioned. But it is only occasionally 
that an ill-conditioned set of equations can be recognized as such by 
inspection. 

A characteristic feature of ill-conditioned equations is that a set of 
values for the unknowns which differs considerably from the solution 
of the equations may, nevertheless, give small residuals for all the equa¬ 
tions. For example, for the equations (8.6) the residuals for certain sets 
of values of Xj, X 2 , X 3 , X 4 are as follows: 

-I-14-6 -7-2 -2-6 -I-31 = -J ?3 = = 0-1 

-1-2-36 -I-0-18 -^0•66 -|-1-21 ^ -R^ = -R 3 = = 0 01 

whereas the exact solution is x^ = Xg = X 3 = X 4 = 1 . Thus in this 
case values of the residuals which are less than 1 /2000 of the values of 
the 6’s in the equations still do not guarantee the accuracy even of the 
first figure in the x’s. This is an extreme case, but it illustrates the need 
for caution in taking the smallness of residuals as a guide to the accuracy 
of the solution when the equations are ill-conditioned. 


t J. Morris, Phil. Mag. (7) 37 (1946), 106. 
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8.13. Normal equations 

For a set of values of (xj, Xj,..., x„), not necessarily a solution of the 
equations, let 2S be written for the sum of the squares of the residuals 
(8.2), that is, ^ ^ ^ ^ ^2 

i 

Then S is a, quadratic function of (xj, Xj,..., which is zero for those 
values of (xj, Xj,..., x„) which form the solution of the equations, and is 
positive for all other sets of values. Hence the determination of the 
solution of these equations is equivalent to finding the set of values of 
(xj, Xj,..., Xn) which make S a minimum. This set is given by 

= 0 (all j) 


dXj 


or 


y = 0. 

4^ sx. 


(8.7) 


(8.7) is 


Now dRJdXj = a-ij = 

where A is the transpose of the matrix A. Hence the set of equations 

2 St = 0, 

i 

2 [(Si«ifcK-«ii^i] = 0; (8.8) 

ik 

or in matrix form (AA)x— Ab — 0. 


or 


The set of equations (8.8) is sometimes called the set of ‘normal 
equations’ corresponding to the original equations (8.1). They are 
derived from the positive definite quadratic form S, and the matrix 
(AA) is necessarily symmetrical; these features are advantageous in 
some methods of earring out the solution numerically. 

On the other hand, the ratio for the normal equations is 

greater than that for the original equations,'}' so that the normal equa¬ 
tions are less well-conditioned than the original equations, and when the 
original equations are at all severely ill-conditioned, the normal equations 
are very much worse. Hence, methods which do not depend on the 
matrix of the coefficients being symmetrical, or on the equations being 
derived from a positive definite quadratic form, are often preferable. 


8.2. Elimination 

A straightforward direct method is one based on successive elimina¬ 
tion of the variables. This is a development of the process of elementary 
algebra with attention given to three points, namely systematic arrange- 

t See Olga Taussky, M.T.A.C. 4 (1960), 111. 
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merit of the work, provision of a current check, and control of rounding 
errors. The points are all important and become more important the 
greater the number of equations and unknowns. 

In elementary algebra emphasis is properly given to the importance 
of verifying that an alleged solution does actually satisfy the equations. 
This final check should always be carried out. But in the evaluation of 
the solution of a set of more than three equations it is hardly sufficient 
since if it fails it gives no indication of the location of the mistake, and 
the whole work has to be repeated with an appreciable probability of 
repeating the same mistake. A current check is required, both to help 
in locating a mistake and to prevent much further work being based on 
erroneous intermediate results. 

To ehminate Xj^ between the tth and jth equations, in which the 
coefficients of are and respectively, we have to multiply the 
equations by and 0 L^ respectively, and add, choosing and so that 

Formally, there is an infinite number of ways of choosing the multipliers 
ot^ and ay, but in practice there are only two ways which are generally 
useful. One is to take one of the multipliers as unity, that is to say to take 


or 

the other is to take 


= 1, —O'ikiv' 

(8.9) 

aj = 

(8.10) 


The division involved in the choice represented by (8.9) will usually 
involve rounding errors in each elimination. Since the results of eliminat¬ 
ing one variable are used later in the elimination of other variables, it is 
important to keep these rounding errors under control, and this is best 
done as follows. One of the multipliers is taken as unity; we choose that 
one of the alternatives (8.9) which makes the modulus of the other 
multipher less than unity. Then at each stage of the eUmination the 
rounding errors from previous stages are always multipUed by numbers 
of modulus less than unity. 

This is a general method. If, however, the coefficients are all fairly 
small integers, it is possible to carry out the elimination without intro¬ 
ducing any rounding errors, by using the choice (8.10) of the multipliers, 
which avoids any division. If this can be done without the coefficients 
becoming inconveniently large it is probably the preferable choice. It 
may be possible to keep the coefficients from becoming large by making 
suitable Unear combinations of the equations, with integral coefficients. 
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in the course of the elimination process; since no rounding off is involved 
there is no loss of numerical accuracy in such a procedure. As a simple 
example, with the equations 

(1) 23x+31t/ = (2) 44x+65y = 

one should not try to eliminate x by multiplying the first equation by 
44 and the second by —23 and adding; the first step should be to form 
the linear combination 2 x (l)-(2): 

(3) = 2x(l)-(2), 2x-Zy = 2b^-b^, 
before continuing the elimination. 


8.21. General elimination process 

To eliminate the variable Xj^ from a set of equations, specified by 
different values of i, we may take one equation, say the jth, and for 
each value of i form 

(ith equation)+(af) X (jth equation), (8.11) 

where is given by the second of formulae (8.9). If the numerical work 
is suitably arranged, it is not necessary to write out the equations in full 
at each stage. It is enough to write down, for each equation, the co¬ 
efficients and the constant term in appropriate columns. A current check 
can be provided by keeping a record, with each equation, of the sum of 
the coefficients and the constant term, and forming the linear combina¬ 
tion expressed by (8.11) not only for the coefficients but also for this 
check sum. The value of this check sum for the ith equation will be 
written sy. that is a i x' 

* = 0f+ ^ %• 

Thus for using the Jth equation to eliminate x^ from the ith equation, 
we have the following scheme: 



Coefficients of 

Constant 

Check 



X^ 

term 

sum 

ith equation 

<^ik 

«« 

bi 


Jth equation 

(^jk 




[«i = 




S^-\-a^Sj 

Result 

= 0 

= «« 

= bi 

f 

= Si 


( 8 . 12 ) 


The check consists in verifying that s'^ calculated as is in agree¬ 

ment (within the tolerance for rounding errors) with the sum of the other 
entries in the corresponding line, namely ^ a^. The jth equation 

here is sometimes called the ‘pivotal equation’, and the coefficient a^j^, 
which is the divisor in the evaluation of the coefficient 
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is called the 'pivotal coefficient’ or 'pivot’ for this elimination; it is the 
coefficient, in the pivotal equation, of the variable to be eliminated. 

Example: To use the first of the three equations; 

3L74xi + 43*6]a:2—16*94a;3+16'94x4 = 4L37, 

6 - 86 a;i +9*81x2 + 7 * 68 x 3 +3*96x4 = 16*81, 
35*85x1-32*92x2+13*81x3 +5*94x4 = 21*84 
to eliminate X 2 from the second and third. 

The pivotal coefficient is the coefficient 43*61 in the first equation, and the 
multipliers are 

= -9*81/43*61 = -0*22495, aa = +32*92/43*61 = 0*75487 


Line no. 
and 

operation 


Coefficient of 


Constant 

term 

Check 

sum 


^1 


^8 


Notes 

(1) 

31-74 

43-61 

-16-94 

16-94 

41-37 

116-72 


(2) 

6-86 

9-81 

7-68 

3-96 

16-81 

46-12 


(3) 

35-85 

-32-92 

13-81 

6-94 

21-84 

44-52 


II 

ft 

X 

— 7-140 

- 9-810 

3-811 

- 3*811 

-9*306 

— 26-256 

= -0-22495 

(6) = (2) + (4) 

-0-280 

0 

11-491 

0-149 

7-504 

18-864 

cross sum 

= 18-864 

(6) = a,X(l) 

23-960 

32-920 

12-877 

-12-877 

31-229 

88-108 

a, = 0*76487 

(7) = (3) + (6) 

69-810 

0 

26-687 

- 6-937 

53*069 

132*628 

cross sum 

= 132*629 


Notes: (i) It is convenient to keep a note, on the left-hand side, of the operations 
carried out to obtain the successive lines of the calculation. This simplifies the 
location and correction of mistakes should any be made, and is also useful if the 
calculation has to be repeated with other values of the constant terms 6 ^. 

(ii) In working, it is best to set each oci on the setting levers or keyboard of a 
machine, then first multiply by the coefficient of the variable which it is required 
to eliminate, to check that has been set correctly, then multiply by the other 
coefficients aji in the pivotal equation. These products can be written down, as in 
the above example in lines (4) and ( 6 ), or the entries in lines (5) and (7) can be 
formed and written down directly. 

(iii) The results of elimination of X 2 from the third equation in the above example 
illustrate a kind of mistake which is not detected by the check explained above. 
The check is satisfied but two entries in line (7) are wrong; in line ( 6 ) the coefficients 
of X 3 and X 4 should read — 12*787 and +12*787 respectively, and the corresponding 
entries inline (7) need correcting accordingly. The failure of the check to indicate 
the presence of the mistakes arises from the fact that the coefficient of X 4 in line ( 6 ) 
is just copied from that of X 3 with a change of sign, and if the latter coefficient is 
wrong, then so is the former and the effects of the two mistakes cancel in the check. 
This shows that special care is necessary when two coefficients in the pivotal 
equation are equal and opposite in sign. It also emphasizes that in the solution 
of a system of equations the current check on the eliminations alone is not sufficient; 
this current check should always be supplemented by a verification that an alleged 
solution does satisfy the original equations. 
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8.22. Evaluation of a solution by elimination 

One way of arranging the solution of a system of equations by the 
elimination process is as follows. Use one equation as pivotal equation 
to eliminate one variable from all the equations. It is convenient to 
consider the variables and equations renumbered (if necessary) so that 
the variable eliminated is Xj, and the equation used as pivotal equation 
at this stage is the first. The result will be a single equation containing 

= K (8-13) 

and a system of {n —1) equations not involving x^, of which a typical 

0110 IS f . t \ \ f L' 

®i2^2 + ®i3^3 + -"+%n^n = (8-14) 


where a\i and b'^ are given by (8.12) with J = 1. Then another variable, 
which can similarly be taken as is eliminated from equations (8.14), 
and so on. 

If the equation used as the pivotal equation to eliminate is chosen 
so that la^jl ^ {i > 1), then none of the multipliers 
in this elimination is greater than 1 in magnitude, a condition which we 
have already seen to be desirable to keep control of rounding errors. 
Similarly, the equation used as the pivotal equation to eliminate x^ 
should be chosen so that |a 22 l ^ Wiz\ > 2), and so on. 

Once an equation has been used as a pivotal equation in the elimina¬ 
tion of one variable in this process, it is left unaltered in the further 
stages of the elimination process. Then this process leads finally to a 
system of equations of which the first is (8.13), the second is that member 
(say with^' = 2) of the set (8.14) which is used as the pivotal equation 
for the elimination of x^ from this set of equations, and so on. This 
system is 


aiia:i+ai2a:2+ai3aJ3+®i4^4+--= 

®22^2”1"®23^34"®24^4”I“-" ~ ^2 

a33^3+«34^4+- = ^3. ©^C. , 


(8.15) 


of which the mth equation contains n —m-j-l of the variables, and the 
coefficients form an upper triangular matrix. 

The last of this set of equations involves only, the last but one 
involves x^^ and x^_i only, the last but two involves x^^, and x^.^ 
only, and so on. This whole system of equations can therefore be solved 
by starting from the last and working backwards so as to determine the 
values of x^, x^_i, x^_^,... in this order, using at each stage the values 
of the x^’s previously obtained. This process is known as ‘back sub¬ 
stitution’. In it, the value of each unknown is determined from that 
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equation which was used as pivotal equation to eliminate this unknown 
in the elimination process. 

In the calculation the usual symptom of an ill-conditioned set of equa¬ 
tions is that the coefficient of in the last equation of the set (8.15) is 
small compared with the values of the coefficients in the equation used 
for the elimination of x^_i; this is illustrated in the following example. 
Another symptom (of which this is a particular case) which may appear 
is that the elimination of one of the unknowns between two equations, 
say equations (A) and {B), by forming the linear combination {A)-\-a{B) 
(with |a| ^ 1), results in an equation in which all the coefficients have 
values which are small compared with their values in equation (A). 


Example: 

To solve the equations 


— 23a;i+ 110 : 3 +a;. 

= 0, 

llXi— 3 X 3 

— 2a;, = 3, 

Xj— 2 x,-fa;, = — 2 , 

Line no, 
and 

operation 


Coefficients of 

6 

Check 

sum Notes 




( 1 ) 

-23 

11 

1 

0 

-11 

( 2 ) 

11 

— 3 

— 2 

3 

9 

( 3 ) 

1 

-2 

1 

-2 

-2 

(4) = «,x(l) 

-11 

6-261 

0-478 

0 

-6-261 a, = 11/23 = 0-47826 

( 6 ) = (2)-K4) 


2261 

-1*622 

3 

3-739 






3-739 cross sum 

( 0 ) = a,X(l) 

-1 

0-478 

0-043 

0 

-0-478 a, = 1/23 = 0-04348 

( 7 ) = ( 3 )-K 0 ) 


-1*622 

1*043 

-2 

-2*478 






—2-479 cross sum 






1-622 

(8) = a,x(6) 


1*622 

-1*026 

2-020 

2-517 a, = = 0-6732 

* 2-261 

(9) = (7)-K8) 



0018 

0-020 

0-038 






0-038 cross sum 

( 10 ) = X, 



1 

1*111 

2*111 

( 11 ) = 1 - 622 X, 



1*622 

1*691 

3-213 

(12) = (6) + (ll) 


2-261 


4-691 

6-962 

( 13 ) = X, 


1 


2-076 

3-076 cross sum checks 

( 14 ) = - 11 X( 13 ) 


-11 


-22-825 

-33-825 

( 16 ) = ( 1 )-|-( 14 ) 






-(10) 

-23 



-23-936 

-46-936 

( 16 ) = X. 

1 



1-041 

2-041 


Final check Residuals 

z, = 1-041 — 23i, + llx,-}-x, = —0-007 — 0 007 

X, «= 2-076 llx, —3x,—2x, = 3-004 -{-0 004 

X, «= 1-111 X, —2x,xx, = -1-998 -+-0 002 
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Notes: (i) The ‘pivots’ are distinguished in this example by being printed in heavy 
tyP® manuscript an underline or ‘box’ could be used), the coefficient of the 
last remaining variable being counted as a ‘pivot’ for this purpose. The largest 
coefficient is chosen as the first pivot, and the result of eliminating x-^ is given by 
lines (5) and (7). In these equations the coefficient of in line (5) is the greatest, 
and this is taken as the next pivot. 

(ii) In line (7) there is a difference of a imit in the last figure between the number 
derived from the other relevant entries in the same column and that derived from 
the cross sum along the row. This is an effect of rounding errors, and occasional 
small discrepancies in the check, such as this, must be expected. In the further 
calculation, the value derived from the cross sum should be used in such cases. 

(iii) If the coefficients are known to have exact integral values, then it is signifi¬ 
cant to keep any number of figures in the calculation. In this example three 
decimals have been kept. Any required accuracy could be attained by keeping 
enough figures, though before the solution is carried out it may be difficult to judge 
how many arc needed to give a specified accuracy in the solution. In the present 
case it might be expected that the three decimals kept should be enough to give 
the solution to 1 per cent., allowing for rounding errors; but as will be seen (under 

(iv) below), the ‘solution’ in this case is not even accurate to two figures. 

If the coefficients were only known to two decimals there would be no significance 
in keeping more than three. If, for example, the coefficient were only known 
to lie in the range —3-00d: 0*01, then the entry 2-261 in line (5) might stand for 
any number in the range 2*251 to 2-271, and the only purpose of keeping even the 
third decimal is to avoid rounding errors accumulating in the second decimal. 

(iv) By taking another decimal in x^ only (x^ — 1-0407) these residuals can be 
reduced to —0-001, +0*001, +0*002 to three decimals, so that at first sight it 
would appear that this solution is correct to at least two decunals. It is not, 
however: the correct solution is x^ = 1, Xg = 2, Xg = 1, and the solution is not 
even correct to one decimal. This is a consequence of the ill-conditioned character 
of the equations. There is a warning of this character of the equations at line (9) of 
the working, at which the coefficient of x^ is very small compared with that in 
equation (7). 

(v) If the coefficients are known to have exact integral values, the value of the 
last pivotal coefficient can sometimes be improved as follows. The product of the 
pivots is the determinant of the coefficients of the original equations,"f and this 
must be an integer if the coefficients are integers. In this example the product of 
the pivots as evaluated is 

-23x2*261x0*018 = -0*94. 

The second factor is certainly not in error by more than 1 in 1000, and the extreme 
possibilities of rounding errors cannot affect the third by more than 10 per cent.; 
so the value of this product lies between —0-84 and — 1-04. But it must be integral, 
and must therefore be — 1; hence the value of the last pivot is 

_1/(_23x 2*261) = 0*0192, 

the fourth decimal being certainly correct. 

This argument must not be used unless the coefficients are known to be integral; 
otherwise it might give a false idea of the accuracy of the solution. 

f A factor (— 1) may be introduced if the order of the variables or of the equations is 
changed in the elirmnation process. 
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(vi) If ^ 1 , ^ 2 , is is an approximate solution, and R^, R^, R^ the residuals obtained 
by putting Xj = ij, then the corrections {xj—ij) to the approximate solution can 
be obtained by solving the equations (8.3) in a similar manner. 

8.23. Alternative arrangement of the elimination process 

In the above arrangement of the elimination process, each pivotal 
equation is left unchanged in later stages of the process. An alternative 
procedure is to use each pivotal equation to eliminate an unknown from 
the pivotal equations previously used, as well as from later equations. 

In the above example, for instance, the pivotal equation (5) can be 
used to eliminate from equation (1) as well as from equation (3). 

This procedure avoids the process of back-substitution, but the 
ehmination process is longer, and the total amount of work involved is 
about the same. 


8.3. Inverse of a matrix by elimination 

One way of inverting a matrix A is to obtain a set of solutions of the 

equations = (8.16) 

i 

for a general set of values of the 6’s. Each step of the elimination pro¬ 
cess consists of forming a linear combination of the left-hand sides of 
the various equations (8.16), so the individual z’s which result from 
the elimination process can be expressed as linear combinations of the 
left-hand sides. If the same linear combinations of the right-hand 
sides are formed, the result is that each Zj- is expressed as a linear com¬ 


bination of the b/s, say 


% = 2 
i 


(8.17) 


or in matrix form x = Cb, where C is the matrix of the coefficients in 
(8.17). But if A is non-singular, there is a unique matrix such that 
X = A~^b; hence the matrix C is just the inverse of A. 

The solution of the equations for general values of the 6's can be 
carried out by a slight extension of the process of § 8.22, using a separate 
column for the coefficient of each 6^. 


Example: To solve the equations 

23xj-f* 11x2*-}“ Xg = 

llXi—3xa — 2 .X 3 = 6 a» 

Xi 2X2-1“X3 — 63 

for general values of 6 ^, 63 , and 63 . The left-hand sides of these equations are the 
same as in the previous example; the right-hand sides have general values. In this 
example the elimination will be carried out by the special process mentioned in 
§ 8.21 as avoiding divisions, and the rounding errors associated with them, in the 
elimination process. 
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Eqitation no. 

Coefficients of 

Coefficients of 

Chech 


and operation 




*1 

6. 

6. 

sum 

Notes 

(1) 

-23 

11 

1 

1 

0 

0 

-10 


(2) 

11 

-3 

_2 

0 

1 

0 

7 


(3) 

1 

-2 

1 

0 

0 

1 

1 


(4) = (l)-(3) 

-24 

13 

0 

1 

0 

-1 

-11 

cross sum checks 

(5) = (2) + 2x(3) 

13 

-7 

0 

0 

1 

2 

9 

cross sum checks 

(6) = (4) + 2x(6) 

2 

-1 

0 

1 

2 

3 

7 

cross sum checks 

(7) = (4)+12x(6) = 

0 

1 

0 

13 

24 

36 

73 

cross sum checks 

(8) = (6) + (7) 

0 

0 

0 

14 

26 

38 

80 


(9) = J(8) = 

1 

0 

0 

7 

• 13 

19 

40 

cross sum checks 

(10) = 2x(7) + (3) 

1 

0 

1 

26 

48 

71 

147 


(11) = (10)-(9) =X3 

0 

0 

1 

19 

35 

62 

107 

cross sum checks 


Xj = 763^-!-1362 "h 
X 2 = 1361 + 2462 + 3563 , 
X3 = 1961 + 3562+6263, 


’-23 

11 

r 

-1 

■ 7 

13 

19‘ 

11 

-3 

-2 

= 

13 

24 

35 

1 

-2 

L 


.19 

35 

52. 


Notes: (i) In this example advantage heis been taken of the simple numerical 
values of the coefficients to lighten the numerical work of the elimination process. 
The particularly simple values of the coefficients of X 3 suggest that this is the 
\inknown to eliminate first. 

(ii) In line ( 6 ) no elimination is carried out, but a linear combination of the 
equations is made so as to keep down the magnitudes of the numbers occurring 
in the calculation. 

(iii) By avoiding division and so keeping the work free from rounding errors, 
the exact solution is obtained without any attention having to be given to the 
number of figures kept at the various stages of the work. Further, the ill-conditioned 
nature of the equations (see note (iv) below) gives no difficulty in obtaining a 
solution. Also the numbers occurring are simple enough in this case for the whole 
calculation to be done without the aid of a desk machine. 

(iv) The large values of the elements of the inverse matrix show why such a poor 
approximation to the solution, as represented by the ‘solution’ obtained in § 8 . 22 , 
gives such small residuals. 

If (^ 1 , ^ 2 .^ 3 ) is an approximation to the solution, and R^y R<^y R^ are the residuals 
obtained on substituting ^2 “ ^3 “ fa i^^^ equations, then the 

corrections to the approximate solution are 

( 3 ^ 1 -fi) = 7i2i + l3i?2+19^3. 

(^ 2 -^ 2 ) = I3i?i + 24i?2 + 35i?3. 

(^a-fa) = 19i?i + 35i?2 + 52i?3, 

so that if = ^3 = 0-^1^ ^a-fa = 1*06; that is, the error in an 

approximate value of X 3 may be over 100 times the residuals in the equations, 
although in the equations this unknown only occurs with coefficients 1 and 2 . 
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8.4. Choleski’s method 

An alternative direct method is one usually ascribed to Choleski,‘f' of 
which there are several variants. It depends on the factorization of the 
matrix A of the coefficients into the product LU of two matrices, of which 
L is lower triangular and U is upper triangular. In such a factorization, 
the diagonal elements of either L or U (but not of both) can be restricted 
to be unity. Then the system of equations 

Ax = LUx = b 

can be written Ly = b, (8.18) 

Ux = y. (8.19) 

Written out, equations (8.18) are 

^11 Vi ~ ^1 ^ 

= ^2 I ( 8 . 20 ) 

^3l2/l"i~^32 2/2"h^33 2/3 ~ ^3 ( 


From these equations, y^, y^, 2 / 3 ,... can be obtained in succession. Equa¬ 
tions (8.19), written out, are 


Wiia:i-fWi2a:2+Wi3X3-l-...+Wi„ar„ = y^ \ 

^22^2'^~^23^3*i~~ Vz I 
. ! ( 8 . 21 ) 

'^n-l,n-l^n—l~^'^n-l,n^n ~ Vn-l I 
Unn^n = Vn ) 

and, once the values of the y^a have been found from (8.20), those of 
the Xj^a can be found in succession from equations (8.21), starting with 
the last and working backwards. 

From equations (8.20) it follows that the y'a are given by 


Vi — ^ilhi \ 

Vz — i^z hiyi)lhz j (8.22) 

Vs — (^3 hiVi ^zzy^hz^ 

and one way of determining them is to evaluate these formulae in 
succession, using in each one the values of the y'a determined from pre¬ 
vious formulae in the sequence. Wilkesf has given an alternative way 
of arranging the work, and has shown how it can be developed to give a 
convenient practical procedure for carrying out the factorization of A 


I See, for example, L. Fox, H, D. Huskey, and J. H. Wilkinson, Quart, J, Mech. 
and Applied Math, 1 (1948), 149 ; A. M. Turing, ibid. 287. For another method of a similar 
type see P. D. Grout, Trans, Amer, Inst. Elect, Eng, 60 (1941), 1235. 

X M. V. Wilkes, Proc, Gamb, Phil, Soc, 52 (1956). 758. 
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in the form A = LU. This uses a scheme similar to that of division of 
one polynomial by another by the method of detached coefficients. 

Consider the result of subtracting from the sequence (6^, 6„) 

such a multiple of the sequence (Z^, Ini) that the first term is reduced 
to zero. This can be laid out as follows: 

^11 hi hi •••) ^1 ^2 ^3 ••• i^ilhi — Vi 

^1 _ hiVi _ ^31^1 

0 62 hiVi ^3 ^31^1 

the ‘quotient’ being and the ‘remainder’ sequence being the numbers 
h'j = bj —ij ^ 2) which would be obtained on the right-hand side 
of equations (8.20) by using the first of these equations to eliminate 
from the rest. Now ‘divide’ this remainder sequence, in the same way, 
by the second column of L, namely the sequence (Z22, Z32,..., Z„2): 

Z22 ^32 •••) ^2 hlVl ^3 ^ZlVl ([^2 ^2lZ/l]/^22 ~ Z/2 

^2 hi Vl _ ^32^2 

0 63 Z3iyi hiVi 

the ‘quotient’ being y^ and the ‘remainder’ sequence the numbers 
bj = bj—ljj^yi—lj 2 ,y 2 . (j ^ 3). This ‘remainder’ is then ‘divided’ by 
the third column of L, and so on. 

In this procedure, successive operations are concerned with successive 
columns of L, whereas in the evaluation of the y’s directly from formulae 
(8.22), each step is concerned with the evaluation of a formula derived 
from a row of L. A check is provided by the result of summing by 
columns. The sum of the equations (8.20) is 

(2 ^n)zZi+(2 = X 

the sum over^' being for the whole set of equations {y — 1 to n) or for the 
first k of them {j = 1 to ky, the sums over k equations {k = 2 to n—\) 
provide current checks. Alternatively, a column-sum check can bo 
kept in the ‘division’ process. A similar treatment can be applied to the 
set of equations (8.21), starting with the last column of U. 

Now consider the factorization A = LU, with U restricted to have 
its diagonal elements unity. That is, we require Z,-^ and such that: 


^11 



"1 

^^J2 

^13 

• 

• 

• 


"ail 

^12 

CO 

• 

• 

hi 

hi 



1 

^23 

• 

• 

• 


®21 

^22 

^23 

• 

* 

hi 

hi hs 




1 

• 

• 

• 


®31 

^32 

^33 

• 

• 


• • ^ 





• 

• 

• - 


_ • 

• 

• 

• 



(8.23) 
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The first column of A gives 

The first column of L is now known. Then from the second column of A 


^ 11 ^ 12 -«i 2 j (3 24) 

^22 ~ ^j2 ^ 2) ; 

of which the first gives u ^2 = ^ 12 /^ 11 ? ^^e unknown element in the second 
column of U; since 1 , the second of equations (8.24) gives 

Z^'2 = ^;l'^12 (i ^ 2), 

the unknown elements in the second column of L. The evaluation of 
these elements can be arranged as a division process: 


^11 


21 


^31 


.) a 


12 


a 


12 


^22 
Z 21 0^12 


^32 
^31 ^13 


(%2/^ll — '^12 


0 


^22 ^21 ^12 


a. 


32 


-Z 31 aj3 

u 


1^22 — 1^32 

The first two columns of L are now known. 
The third column of A gives the equations 


^11 '^13 


= a 
= a 


13 


23 


(8.25) 


^ 21 '^13*^~ ^22^^23 

^;1^13‘^~^;2'^23"i“^;3'^33 ~ ij ^ 3) ^ 

of which the first two give 

'^13 ~ ^13/^11 

and ^23 ~ (^23 ^ 21 ^ 13 )/^ 22 ’ 

the two unknown elements in the third column of U; since ^3 = 1 , the 
third of equations (8.25) gives 

^j2'^23j 

the unknown elements in the third column of L. The calculation can 
again be arranged like a division, the two 'di\dsors’ being the two 
columns of L previously determined: 


^11 ^21 ^31 •••) ^ 


13 


a 


23 


a 


13 


Zo-i W- 


21 ^13 


(X 33 ... (^ 3/^11 — '^13 

^31'^13 


^22 ^32 •••) ^23 ^ 21 ^ 


13 


^23 ^21 ^13 


^33 ^31 ^13 

^32 "^23 




23 4l '^13 


I 


= U 


23 


22 


%3 ^ 31^13 ^ 32^23 


= I 


33 
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The procedure can now be applied to the further columns of A in suc¬ 
cession. In each case the ‘divisor’ sequences are the successive columns 
of L already evaluated, the ‘quotients’ are the unknown elements in the 
next column of U, and the ‘remainder’ sequence consists of the unknown 
elements in the next column of L. 


The only divisions of numbers required are divisions by the diagonal 
elements of L. Now the determinant of a triangular matrix is the 
product of its diagonal elements, and so is unity for U. Hence the 
determinant of L is equal to that of A, which, as stated in § 8.1, is sup¬ 
posed to be non-zero so that the equation Ax = b has a unique solution. 
Hence no diagonal element of L is zero, and no step of the calculation calls 
for a division by zero. When the equations are ill-conditioned, however, 
one diagonal element of L (or more) will be small compared with the 
others, and, unless an adequate number of digits have been carried in 
the intermediate working, the values obtained may be considerably 
influenced by rounding errors. 


Example: To factorize the matrix A = 



The first column of L is the same as that of A. Since in this example Ojj = 0 it 
follows that Mj 2 = 0 and the second coliunn of L consists of the other eleiuents 
of the second column of A. Hence 


L = 

■ 2 

, u = 

"1 0 X x‘ 


-1 1 


1 X X 


4 0 X 


1 X 


2—1 X X. 


1. 


the X’s standing for elements still unknown, and the blanks for elements known 


to be zero. 

For the third columns of U and L, the process of ‘division’ of the thii’d column 
of A by the first and second columns of L gives 


2-142) 
1 0 - 1 ) 


so that 


L = 


-4 5 -5 -7 (-2' 

-h2 —8 —4 1 third column of U 


3 

3 


3 -3 (3 

0 -3 


3 0 

third column of L 


■ 2 

II 

‘1 0 -2 x“ 

-1 1 


1 3 X 

4 0 3 


1 X 

1 - 

1 

h-' 

o 

X 


1 . 


X’s indicating elements still unknown. 
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Then a similar treatment applied to the fourth column of A gives 

2 -1 4 2) 6 -2 6 10 (3 

6 -3 12 6 

10—1) +1—6 4(1 > fourth column of U 

1 0 —1 

3 0) -6 6 {-2) 

-6 0 

5 = lu 

so that finally 


L = 

■ 2 

, U == 

"1 0 -2 3' 


-1 1 


1 3 1 


4 0 3 


1 —2 


_ 2 -1 0 5_ 


1. 


Note: To illustrate the procedure, an example has been taken in which the elements 
of L and U are integral. This simplifies the numerical work, but is not important 
in principle. 

Other methods of carrying out the factorization A = LU and for 
solving the resulting equations have been proposed.f Some of these 
methods are designed to reduce the number of intermediate quantities 
which have to be written down in the course of the calculation; this is a 
matter of some importance in the treatment of large matrices. On the 
other hand, in methods designed wdth tliis object, too little is written 
down to indicate how the numbers which are written down were obtained, 
and this makes diagnosis of mistakes difficult; the use of row and column 
sums gives a good indication of freedom from mistakes when none have 
been made, but the checks are too few to be of much help in identifying 
or locating a mistake if one is indicated. 

The absence of intermediate results to make it clear how each number 
in the course of the calculation is obtained means that the computer 
must depend on his memory for the required sequence of operations. 
This sequence is slightly different for every one of the elements of L 
and of U, and though it might become familiar enough to anyone who 
had much work of this kind to do, the writer’s experience is that such a 
method is too complicated to be satisfactory for occasional use; it seems 
more suitable for the professional expert and the specialist rather than 
for the occasional user. In Wilkes’s process given above, more may be 
written down, but the sequence of numerical operations is more apparent, 
and so easier to follow and to remember. 

t See, for example, L. Fox, Joum. Roy. Slot. Soc., Ser. B, 12 (1952), 120. 
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8.41. Inverse of a matrix by Choleski’s method 

The inverse of a lower triangular matrix L can be found by taking 
the identity LL“i = I, or in expanded form, with C for 


o 

0 

0 . 

• 

• 


"cll 

C12 

^13 

• 


^22 

0 

0 . 

• 

• 


C21 

C22 

C23 

• 

• 

^32 

^33 

0 . 

• 

• 


^31 

^Z2 

^33 

• 

• 













"1 0 0 0 . 

0 10 0 . 

0 0 10 . 


and working through the unit matrix column by column. 

From the jth column of the unit matrix we have 

III — ^ij 

^ 21 ^ 1 / 4 *^ 22^27 = ^ 2 / 

^31 ^174*^32 ^2? 4* ^33 ^37' ~ ^3/ 

(where = 1 if_; = i and = 0 if 4 ^ i). This is a set of equations of the 
form (8.20); the solution can be obtained by the ‘division’ procedure 
already explained for such a set of equations; this has to be done for 
each value ofSince 8 ^^ = 0 for i ^ j, it follows that = 0 for i < j, 
that is, that L“^ is also lower triangular. This can also be seen from 
formulae (8.22), from which it follows that if Ly = b, then is a linear 
combination of the 6^’s for i ^ j only. The inverse of an upper triangular 
matrix can be found in a similar way, starting from the last column of the 
unit matrix. 

If A = LU, 

then A“^ = 

so that a matrix can be inverted by factorizing it as a product LU, 
inverting U and L separately, and finally multiplying them to form A"^. 

8.5. Relaxation method 

An indirect method which is very powerful in some cases is one called 
the ‘relaxation’ method. It was originally developed by Southwellf 
for application in problems of structural engineering, and some of the 
terminology of the method is derived from this particular application. 
But its range of application is much wider. 

t R. V. Southwell, Proc. Roy. Soc. A, 151 (1935), 56; see also ibid. 184 (1945), 253 ; 
Relaxation Metfiods in Engineering Science (Oxford, 1940). 
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For any approximation Xj = to the solution of equations (8.1), the 
‘residuals’ of the equations are defined by (8.2). For the solution of the 
equations, all the residuals are 0. In the relaxation method attention 
is concentrated on the residuals, and the method consists in making 
changes in the x’s in a systematic manner so as to reduce the magnitudes 
of the residuals to neghgible amounts. 

The first process is to draw up an ‘operations table’ giving the change 
of each residual for a unit change of each single Xj. Then a set of initial 
values of x^ is taken and the residuals calculated, and changes of the 
XjS then made in such a way as to decrease the residuals; the steps of 
this part of the work are recorded in a ‘relaxation table’ in which the 
changes of the x’s and the resulting tokil residuals are recorded. 


Example: To find, correct to two decimal places, the solution of the equations 


9x^ — 2x2-\-x^ = 50^ 
Xi -\-5 X 2 — 30^3 =18 

— 20:1 + 2x2 + 70:3 = 19 , 


(8.26) 


For the first equation the residual for any trial values of X 2 , x^ is 

Ej ^ — 9 xi — 20:2+0:3— 50 . 

Hence for a change diX^ — I of alone, the change of E^ is Ai?i = 9; similarly 
for a change Ao:2 = 1 of 0:2 alone, the change of Ei is Ai^i = — 2 , and for a change 
Ao :3 = 1 of 0:3 alone, the change of is Ai?i = +1. These values of Ai?i are 
entered in the first column of the operations table. 

Similarly, the residual for the second equation is 

7^2 — 0^1 “f" 50/2 30:3 ”” 18 , 


and for the same changes Ao:i, Ao: 2 , Ao: 3 , the changes in E 2 are Aii ^2 = + 

— 3 respectively, and so on. Thus in this arrangement of the work, the matrix of 
these entries in the operations table is the transpose of the matrix of the coeffi¬ 
cients in the equation. The working is shown on the opposite page. 

Notes: (i) In this example, a change of x^ affects E^ mainly, and i? 2 , E^ to 
a smaller extent, and similarly for changes Ax 2 and Aa: 3 . A change Ax^, made in 
such a way as to reduce \E^\ considerably, is called a ‘relaxation’ of x^, and the 
relaxation process consists of making a sequence of such relaxations. There is 
clearly no point in choosing Ax^ so as to make exactly zero, since E^ will be 
affected by subsequent relaxations A:c 2 and Aa: 3 . It will usually be adequate to take 
single-digit numbers for the relaxations; this greatly lightens the numerical work, 
and makes it possible to carry out the greater part of it mentally and speedily. 

(ii) The relaxation table begins with any trial set of values of x^, X 2 , x^ and the 
corresponding residuals. The simplest first trial set is Xi = X 2 = x^ — 0, and the 
residuals are then just the negatives of the constant terms in the equations. 

In this case the residual E^ is the greatest, and a large part of this can be removed, 
with smaller changes in the other residuals, by a relaxation t^x^ = 5 . The resulting 
changes in all the residuals are given by multiplying line ( 1 ) of the operations table 
by 5. In the example on p. 187, these changes arc shown in brackets; they would 
not be written down in actual working; each would bo evaluated mentally and 
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Axj 

Ax 2 

AX 3 


Ai?, 

Ai ?3 

Notes 

Operations 

1 

0 

0 

9 

1 

-2 

Line(i) 

table 

0 

1 

0 

-2 

5 

2 

Line (ii) 


0 

0 

1 

1 

-3 

7 

Line(iii) 






i?3 




= 0 

X 2 = 0 

11 

0 

-50 

-18 

-19 


Relaxation 

6 

( 0 ) 

( 0 ) 

(45) 

(5)(- 

- 10 ) 

Line(i)X 5 

table 




-5 

-13 

-29 



( 0 ) 

( 0 ) 

4 

-1 

-25 

-1 



( 0 ) 

5 

( 0 ) 

-11 

0 

9 



1 

( 0 ) 

( 0 ) 

-2 

1 

7 



( 0 ) 

( 0 ) 

-1 

-3 

4 

0 



( 0 ) 

— 1 

( 0 ) 

— 1 

— 1 

-2 



x^ = 6 

11 

! 

X 3 = 3 

— 1 

— 1 

-2 

Cheek 




XlO 






lOxi = 60 

10 x 2 = 40 

10 x 3 = 30 

-10 

-10 

-20 





3 

-7 

-19 

1 




4 


— 15 

1 

9 



2 



3 

3 

5 





— 1 

2 

6 

-2 




— 1 


4 

1 

— 4 



-1 



-5 

0 

-2 



10^1 = 61 

lOx, = 43 

10a:, = 32 

— 5 

0 

-2 

Check 




X 10 






610 

430 

320 

-50 

0 

-20 



5 



-5 

5 

-30 





4 

-1 

-7 

-2 




1 


-3 

-2 

0 


lOOari = 615 lOOa;, = 431 lOOarj = 324 

-3 

-2 

0 

Cheek 




X 10 






6150 

4310 

3240 

-30 

-20 

0 




4 


-38 

0 

8 



4 



-2 

4 

0 




-1 


0 

-1 

-2 



6154 

4313 

3240 

0 

-1 

-2 

Check 


added to the previous value of the corresponding residual, and only the new residual 
would be written down. For example, the value = 9 in the operations table, 
multiplied by = 5, gives 45, which added to the old (= *— 50) gives the new 
jRj (= “5) and only this is written down. In subsequent lines in the relaxation 
table the only entries in the Rj columns are those which would be written down 
in actual working. Similarly, the bracketed zeros in the Axy columns might be 
omitted, as they have been later in the relaxation table. 

After the first relaxation Axj = 5 the largest residual is = —29 and most of 
this is removed by a further relaxation Ax, = 4; then R^ is greatest and can be 
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reduced to zero by Ax 2 = 6 . The contributions to -Rj from these relaxations Ax, 
and Aa :2 have been such that |-Ri | can be reduced substantially by a further relaxa¬ 
tion Axi = 1 , and the procedure continues until the residuals are so small that they 
cannot be improved without making relaxations of magnitude smaller than unity. 

At this stage it is convenient to avoid decimal points by multiplying the entries 
in the relaxation table by 10 . But before doing this it is advisable to check that 
the calculation has been carried out correctly so far. This is done by adding up the 
changes Ax^ and adding the result to the initial value taken (here zero), and 
similarly for Xj, X 3 . The resulting values of Xj, Xg, X 3 are then substituted in the 
equations and the residuals calculated; they should agree with the last line so far 
obtained in the operations table. If they do not, there is no need to go back and 
look for the mistake; these values of Xj, Xg, X 3 form as good a set of trial values as 
the set ( 0 , 0 , 0 ) actually used—indeed they are probably much better—and can 
be used as the starting values of a further calculation. 

(iii) When a set of small residuals has been obtained and checked, the values of 
X and the residuals can be multiplied by 10 and the process continued, and this 
operation can be repeated as often as required. In this example it is repeated until 

It gives the result lOOO(a:i,Xjj, x^) = (6164,4313,3240) 
or, to two decimals, = (6*15,4-31, 3-24). 

It will be seen that although the number of operations may be large, each is very 
simple and, except for the checking operations, each involves only numbers of two 
digits, and often only of one digit. 

8.51. Group relaxations 

There is no need to restrict oneself to relaxations of the variables 
singly. In this example a relaxation Axg makes appreciable contributions 
to Ri and R^, and the process of reducing the residuals would clearly 
be quicker and easier if we could make correlated changes in the variables 
in such a way as to affect only one residual considerably. It is often 
possible by trial to find linear combinations of the variables with this 
property. In this example we have: 



Axi 


Axj 

AJ?i 

Ai 2 j 

Ai2, 

^3 

0 

1 

2 

0 

— 1 

16 


0 

4 

— 1 

-9 

23 

1 

^3 

1 

4 

-1 

0 

24 

-1 


A multiple of Xg can clearly be used to reduce R^ without affecting R^ 
and with only quite a small effect on R^, and similarly a multiple of 
can be used to reduce R^,. Relaxations which are multiples of Xg or Xg 
are called ‘group relaxations’ by Southwell. Use of them corresponds 
to making a linear transformation of the variables so that in the trans¬ 
formed variables the non-diagonal coefficients in the equations are small 
compared with the diagonal coefficients. But we do not have to carry 
out the transformation formally by introducing the new variables and 
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expressing the equations in terms of them. The process of carrying out 
the numerical work makes this transformation for us. 

Use of these group relaxations does not, of course, preclude us from 
using relaxations of a single variable if the values of the residuals indicate 


that such a procedure would be 

appropriate. 




Example 


Aa:* 

Axj 

Ai?i 

Ai?g 

Ai2, 


Operations 

1 

0 

0 

9 

1 

-2 


table 

0 

1 

0 

-2 

5 

2 



0 

0 

1 

1 

-3 

7 



1 

4 

-1 

0 

24 

— 1 



0 

1 

2 

0 

-1 

16 


Relaxation 


X 2 

2^3 

R, 

R 2 



table 

0 

0 

0 

-50 

-18 

-19 



5 



-5 

-13 

-29 


2 X 3 


2 

4 

— 5 

-15 

3 


X3 

1 

4 

-1 

-5 

9 

2 



6 

6 

3 

-5 

9 

2 

Check 




X 10 






60 

60 

30 

-50 

90 

20 


- 4 X 3 

-4 

-16 

4 

-50 

-6 

24 



6 



-5 

-1 

14 


-^3 


-1 

-2 

-5 

0 

-2 



61 

43 

32 

-5 

0 

-2 

Check 




X 10 






610 

430 

320 

-50 

0 

-20 



6 



4 

6 

-32 


2 X 3 


2 

4 

4 

4 

0 



616 

432 

324 

4 

4 

0 

Check 




X 10 






6160 

4320 

3240 

40 

40 

0 


- 2 X 3 

-2 

-8 

2 

40 

-8 

2 



-4 



4 

-12 

10 





-2 

2 

-6 

-4 




1 


0 

-1 

-2 



6154 

4313 

3240 

0 

— 1 

-2 

Check 


6-154 

4-313 

3-240 

Solution (to 3 decimals) 


8.52. Use and limitations of the relaxation method 

Templef has shown that if the simultaneous equations are derived 
from a positive definite quadratic form, then the relaxation process 
f G. Temple, Proc. Roy. Soc. A, 169 (1938), 476. 
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formally converges. However, this is neither a necessary nor a sufficient 
condition for it to be a practicable process in actual numerical work; it 
is quite impracticable for the ill-conditioned equations (8.6) which can 
be derived from a positive definite quadratic form, whereas it is quite 
practicable for equations (8.26) which are not so derivable. What 
matters much more is that it should be easy to find a set of relaxations, 
either of individual variables or group relaxations, each of which has a 
considerablj'^ greater effect on one of the residuals than on any other. 

Although the relaxation method is one of successive approximation, 
there is no limit in principle to the accuracy to which the solution can 
be taken. In the structural engineering context for which the method 
was originally devised by Southwellf there is no significance in carrying 
the solution beyond a certain hmited degree of accuracy, and its first 
presentation in this context appears to have given the impression that 
it is fundamentally approximate in character; but it is no more so than 
any other numerical process which is subject to rounding errors. When 
it is used in contexts in which no approximation is involved in the 
equations to which it is applied, it may be possible, and significant, to 
carry the calculation through to a relatively high accuracy; and because 
of the simplicity of the process this may be the easiest way of deriving 
results to such accuracy. 

For example, the recurrence relation for the Bessel functions 
‘4+i(^)-(2W^)‘4(^)+‘^n-i(a^) = 0 

provides a set of simultaneous hnear equations for JJ^x) as a function 
of 11 for given x\ and given, say, JJ^x) for n = N ^ x and the condition 
Jn{x) 0 as w -> 00, it is possible to solve these equations to any accuracy 
required by an appheation of the relaxation process. The process is a 
simple and quick one, and Fox| has recorded that starting only from 
Jio(lO), he has obtained eighteen-decimal values of t4(10), for values of 
n from 11 to the value {n = 37) at which *4(10) < 10“^®, without 
difficulty. This process is most effective for calculating J^ix) for n > x, 
which is just the range over which the use of the recurrence relation to 
evaluate «4+i(^) from J^ix) and *4-i(^) becomes unsatisfactory (see 
§ 11 . 3 ). 

In one particular application, namely to the solution of ordinary 
differential equations with two-point boundary conditions J or of partial 
differential equations of elliptic type (§ 10.6), the relaxation process is 

t R. V. Southwell, Proc. Roy, Soc, A, 151 (1935), 56. 

X L. Fox, Proc. Camb. Phil. Soc. 45 (1949), 50. 
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combined with the use of finite-difference approximations to derivatives, 
and this is perhaps part of the reason why the relaxation process has 
come to be regarded as essentially approximate. But this is a mistaken 
idea. The approximation here is in the reduction of the differential 
equation to a set of simultaneous algebraic equations by use of finite 
differences. This approximation is involved whatever numerical process 
is used for the evaluation of a solution of these simultaneous equations, 
and the relaxation process, if used, is quite distinct from this approxima¬ 
tion. 

If a number of sets of equations with the same set of coefficients but 
with different values of (6i,..., b^) have to be solved, no advantage can be 
taken of this in the relaxation process, and for this reason this process 
is not well adapted to the inversion of a matrix. 

The main difficulty hkely to arise in using the process is slow conver¬ 
gence, and this will be most hkely to occur when many of the coefficients 
in the equations are of the same order of magnitude, so that a relaxation 
of one of the unknowns makes changes of similar magnitude in a number 
of residuals. This situation is likely to occur when the equations are 
ill-conditioned, but is not necessarily a symptom of this condition. 

8.6. Linear differential equations and linear simultaneous 

equations 

Consider the linear differential equation 

y" = ( 8 - 27 ) 

with two-point boundary conditions, y = y^ at x = y = y^ 

X — x^ = XQ-\-n{hx). To the approximation in which {^xYy" can be 
replaced by this differential equation is equivalent to the set of 
algebraic equations ^ {ix)y,y,+g,l 

or = (8*)^,. (8.28) 

for 1 J < n — 1, with and y„ given. This is a set of linear algebraic 
equations for {n —1) unknowns, and could be solved numerically by any 
of the methods for the solution of such equations. 

A convenient process in many cases is one that has been suggested 
and developed by Thomas and by Fox.f It is a version of the Choleski 

I See L. Fox and H. H. Robertson, Proceedings of a Symposium on Automatic Digital 
Computation, N.P.L. 1953 (H.M.S.O., 1954), ch. 19, section on ‘Boundary Value Prob* 
lems’; also L. Fox, The Numerical Solution of Two-point Boundary Problems in Ordinary 
Differential Equations (Clarendon Press, 1957), ch. 3, § 29. As far as I am aware, Thomas’s 
work on this process has not been published; see the Preface to Fox’s book. 
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method (§ 8.4), which in this case is very simple because of the specially 
simple forms of the matrices concerned. 

Equations (8.28), arranged with the known quantities on the right- 
hand side, are 

—[2 + (8a:)2/i]yi+y2 = 

[2+(8a:)2/,.]y,.+y^+i = {^xYg^ (2 < j ^ n—2) . (8.29) 

yn-2-[2 + (Sa:)2/„_i]y„_i = 

For short, let <f)j = {8x)Yj, and 

Cl = —c,- = (8a:)2^^^ (2 < j < 71-2), = -y„ + (Sa:)2^,„_i; 

also let y = (^i, y 2 >--> 2 /n)» c = (Ci,C 2 ,...,c„). Then equations (8.29), in 


matrix form, are 


Ay = c. 


(8.30) 


where A is the matrix 






A = 

~(2 + ^i) 

1 

0 

0 . 

. 0 

0 

0 


1 

— (2 + ^ 2 ) 

1 

0 . 

. 0 

0 

0 


0 

1 

— ( 2 +^ 3 ) 

1 

. 0 

0 

0 


0 

0 

0 

0 . 

. 1 

— (2-1-^„_2) 

1 


0 

0 

0 

0 . 

. 0 

1 



It can easily be verified that for this matrix A the lower and upper 


L = 


ices of § 8.4 

are 



■ 1 

0 

0 . . 

0 

0 

-k 

1 

0 . . 

0 

0 

0 

k 

1 . . 

0 

0 

0 

0 

0 . . 

k-Z 

1 

0 

0 

0 . . 

0 

-In 


(8.31) 


U = 


- 111 , 1 

0 —l/Zj 

0 0 


0 

1 

-1/^3 


0 

0 

0 


0 

0 

0 


-l/^n-2 


1 


0 




0 0 0 

, 0 0 0 

where 

Ijl, = (2+,^J, = (2+,^,.) (2 < j < 71-1) 

so that 

I, = lj{2+cf>,), Z,. = l/(2-f,^.-Z._,) (2 < j < 71-1). (8.32) 

From these relations the Ij’s can be evaluated in succession, in order 
of increasing j. 
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Also equation (8.30) becomes LUy = c, so that if z = — Uy, then 
Lz = —c, and for the matrix L given by (8.31) it follows that 

2 i = —Cl, Zj = (2 ^ w—1), (8.33) 

from which the z^’s can be evaluated in succession, in the order of j 
increasing; then from the equation z = —Uy it follows that 

Vn-l = Vj = hiVi+l+^j) (1 < ^1-2) (8.34) 

from w'hich the y^’s can be evaluated in succession, in the order of j 
decreasing. 

The process is a direct one, and, if solutions of (8.30) are required 
for the same function/(.r) and several different vectors c, the matrix A, 
and so the values of the are the same for all vectors c, so the evalua¬ 
tion of the IjS only has to be done once. This may be necessary if it is 
required to improve the approximation to the solution of the differential 
equation (8.27) by including higher orders of differences in the replace¬ 
ment of derivatives by differences. The process is easily adaptable to 
second-order hnear differential equations containing a first-derivative 
term, and was indeed formulated by Thomas and by Fox for this more 
general case, but it is particularly simple for equations like (8.27) for 
which the first derivative is absent. Also higher-order differences can be 
included in the replacement of y" by finite differences, as follows. 

The set of equations (8.28) has been obtained by replacing the deriva¬ 
tive in (8.27) by the simplest finite-difference approximation to it. In 
the solution of the equations in the approximate form (8.28), y" is never 
evaluated; so in taking higher orders of differences into account, it is 
more convenient to use those of y than those of y" . The next approxima¬ 
tion to y"j is given by 

use of this approximation in (8.27) gives 

y,-+i-[2+(Sa:)2/,.]y,-fy,._i = (8-35) 

This set of equations should be solved by an iterative process, the values 
of 8^y on the right-hand side in one iteration being obtained from the 
results of the previous iteration; higher orders of differences of y can be 
included on the right-hand side of (8,35) if appreciable. One should 
not try to solve it by expressing hhjj in terms of function values and 
treating (8.35) as a recurrence relation between five successive values 
of y; this would be equivalent to replacing equation (8.27) by a fourth- 
order equation and would probably introduce spurious ‘solutions’. 

o 
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The terms (and higher difference terms if appreciable) can be 

incorporated in the c/s. 

Example: To find an approximation to the solution of the equation 

2/" = 1 (8.36) 

with 2 / = 0 at X = ±4. 

From the symmetry of the equation and the boundary conditions, it follows that 
y is an even function of x. Hence wc need only consider the range x = 0 to 4, and 
impose a condition of symmetry about x = 0. 

Let us take Sx = ^ and reckon j from ^ = 0 at x = 0. Then equation (8.28) 

becomes for 1 < i < 7. (8.37) 

and the condition of symmetry, y_^ — 2 / 1 , gives for ^ = 0 the equation 

■^ 2/0 + 2 /! = —|:+2\S‘*2/o; (8.38) 

so that 

<f>j = h = Co = -i + AS^i/o, c,- = -i+iW2/; (1 < i < 7); (8.39) 

and equations (8.32), (8.33) apply for 1 < y < 7. 

The working can be arranged as shown on p. 195; some intermediate results, 
which it would not be necessary to write down if working with a desk machine, 
are included to show the sequence of operations. 

In this working, the calculation is split into three sections by heavy lines. The 
first section, to the left of the first heavy line, is concerned with the calculation of 
the /j’s. The values of (2+^y) are filled in first from the formula for <f>jy then the 
Zj-’s calculated in succession from formulae (8.32). The second section is con¬ 
cerned with a first approximation to 2/y, neglecting the fourth-difference terms in 
(8.37), (8.38). The zfs are calculated first, by working downwards through the 
columns to the left of the thin line; each value of is multiplied by the value of 
in the same line, and the product is written in the next lower line in the column 
headed lj_iZj_^, and added to the ( —Cy) in that line to give the next Zj. 

The next three columns are concerned with the evaluation of 2 /; starting from 
2/8 = 0 and working upwards \ as each y^ is calculated it is entered in the next higher 
line in the column headed 2/;+i and added to the z; in that line; the sum is then 
multiplied by 1 ^ in that line. 

The next three columns are concerned with the evaluation of the h^y terms in 
formula (8.39). The third section of the calculation is a repetition of the procedure 
of the second section with the h^y^ contributions to the c^’s included. 

Notes: (i) The symmetry of the solution about y — 0 has been used in evaluation 
8 ^yQ and 8 ^ 2 / 0 - 

(ii) Since y is required to be zero at x = 4 (y = 8), y"(^) = — 1, so 

S^2/8 = (Sx)22/"'(4) + 0(8x)'* = —'0*250 approximately. 

This value (enclosed in brackets) has been used to give a value of 8 '* 2 / 7 * 

(iii) The calculation could be repeated with values of h^y derived from the 
second iteration. 

(iv) Smaller intervals (8x) should be used if greater accuracy in the results were 
required. 

(v) This process may not be satisfactory when the function/ in equation (8.27) 
is negative over a considerable range of x. 
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Entries in columns headed calculated from formulae (8.34). 
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If a boundary condition involves the derivative y', it can best be 
handled by imagining the range of x extended one interval beyond the 
actual boundary, and ehminating the value of y at the virtual external 
point. Suppose, for example, that the boundary condition at a: = 0 is 

“ 2 / 0 +^ 2/0 = y> 


a, /S, and y being given, is not now known, and an extra equation is 
required to determine it. 

Let suffix — 1 refer to the external point. Then for x = equation 
(8.28) becomes 

and, to the same order of accuracy, we have 

y'o = (2/i-y-i)/2Sa:. 


Ehmination of y^ and between these three equations gives a relation 
between y^ and y^ which takes the place of the first of equations (8.29); 
and the range of j to which the second of equations (8.29) applies now 
extends to j = 1. 

A similar treatment applies to a boundary condition at x^ involving a 
derivative, and to conditions at a point x^, between x^ and a:„, at which 
f(x) is discontinuous but y and y' continuous, or at which hx changes. 

An alternative procedure for the solution of equations (8.29) is a 
relaxation process.f It is best to start such a process with quite a coarse 
subdivision of the range over which the solution is wanted (such as 
hx — 1 in the above example) and to divide it further as the approxima¬ 
tion improves. Beyond a certain stage of the subdivision, depending on 
the behaviour of y as a function of x and the accuracy required in the 
solution, it is often better, as suggested by Fox,t to keep some higher 
differences in the replacement of y" by finite differences. 


8.7. Characteristic values and vectors of a matrix 

The ‘characteristic values’ (also called ‘latent roots’) A of a matrix 
are those numbers for which the system of equations 

Ax = Ax (8.40) 

has a solution other than x = 0, and the solution x in such a case is called 
a ‘characteristic vector’ (or ‘latent vector’) of the matrix or a ‘charac¬ 
teristic solution’ of the equations. 

*f L. Fox, Proc, Gamb. Phil, Soc, 45 (1949), 50, and The Numerical Solution of Two^ 
point Boundary Problems in Ordinary Differential Equations (Clarendon Press, 1957), ch. 3, 
§§ 20-25. 
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Written out, the system of equations (8.40) is 

(®11 = 0 

® 21 ^ 1 “ 1“(®22 — ■^)^ 2 “ i “^ 23 ^ 3 “ l "-” = 0 

®31 ^ 1 “ 1“®32 ^ 2 “ 1 “ (*^33 = 0 

and the condition that this set of equations should have a non-trivial 
solution is that the determinant of its coefficients should be zero. An 
algebraic equation of the 72,th degree in A can be obtained by multiplying 
out the determinant, though for n > 2 it is seldom that this provides 
the easiest or quickest way of evaluating characteristic values numeri¬ 
cally. But this formal equation gives three results which are useful in 
numerical work. The equation is 

= 0, (8.42) 

where D is the determinant of the matrix. Hence 

(i) the equation for A has n roots (multiple roots, if any, being counted 
according to their multiplicity); 

(ii) the sura of the roots is the sum of the diagonal elements of the 
matrix; 

(iii) the product of the roots is the determinant of the matrix. 

The characteristic values A, arranged in order of decreasing |A|, will be 
written A<^), A<^\ A<®),..., and the solution Xj for A = will be written x'fK 
Since equations (8.41) are homogeneous, each solution is undeter¬ 
mined to the extent of an arbitrary multiplying factor. In the formal 
treatment of these equations it is convenient to regard this multiplying 

constant as chosen so that 2 ~ such a solution is called 

} 

‘normalized’. But for numerical work it is usually more convenient to 
take it so that the greatest in magnitude of the x^'s is unity. 

A commonly occurring type of matrix is a symmetrical matrix, and 
symmetrical matrices have several properties which aie useful in 
numerical calculations. The characteristic values A are all real, and the 
characteristic vectors for different values of A are orthogonal, that is to 
say, if A(P) 7^ A(«), then ^ 43^ 

J 

and if any value of A is an m-fold root of the equation (8.42), then m 
mutually orthogonal solutions Xj can be found for this value of A. Hence 
it is no restriction to take all the characteristic vectors as mutually 
orthogonal, and it will be assumed that they are so taken. 
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Further, the behaviour of the quantity 

A = ( ^ Xfc) 2 S-) 

as a function of the vector x has a property which makes it useful in 
the approximate calculation of characteristic values. In matrix notation 
this quantity can be written 

A = (x'Ax)/(x'Ix), (8.45) 

where I is the unit matrix, and x' is the row vector corresponding to the 
column vector x. 

Consider first the result of evaluating A for one of the characteristic 
vectors of the matrix, say for x = From the definition of a charac¬ 
teristic vector, it follows that Ax^*’) = and hence that the value 

of A obtained is just A = the corresponding characteristic value, 
-hat IS ^(p) ^ (x(pyAx^P^)/(x(p^'lx(P^). 

Now consider the value of A evaluated for a vector x differing slightly 
from x^P\ say for x = From (8.45), its difference from the value 

is given by (A_A(p))(x'Ix) = x'(A-A(p)I)x. (8.46) 

Since x^^^ is a characteristic vector with characteristic value A^^^ it 
follows that (A—A^Pd)x^p) = 0, and hence 

(A-A(P>I)x = (A-A(P)I)^. (8.47) 

Since A is symmetrical, so is A—A^^d; hence from (8.46) and (8.47) 

(A-A(p))(x'Ix) = x'(A-A(p)I)^ 

= ^'(A—A^Pd)x (since A—A^^d is symmetrical) 

= ^'(A—A^^d)^ (by a second use of (8.47)). 

Hence A differs from by a quantity which is second-order in in 
other words, the quantity A defined by (8.44) is stationary for small 
variations of the vector x from a characteristic vector. 

Hence from a fair approximation to a characteristic vector a relatively 
good approximation to a characteristic value can be obtained by 
evaluating formula (8.44), and from a good approximation to a charac¬ 
teristic vector a much better approximation to a characteristic value 
can be obtained. In application to the equations of vibrating systems 
of several degrees of freedom, characteristic values represent squares of 
frequencies of normal modes of vibration; an account of the use of 
formula (8.44) to determine characteristic values in this context, and 
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developments of it, is given in Mayleigh’s Principle, by Temple and 
Bickley.f 

It is sometimes convenient to express a symmetrical matrix in terms 
of its characteristic vectors and characteristic values. Let x'x stand for 
the matrix whose {j, A;)th element is XjXj.. Then if the characteristic 
vectors x<p) are normalized, the required expression is 


A = 2 


if, as is often more convenient for numerical work, they are not nor¬ 
malized, we have ^ ^ [A<P>x<P)'x<py J (8.48) 


The characteristic vectors of the inverse of a symmetrical matrix 

A are the same as those of A itself, and the characteristic values of A“^ 
are the reciprocals of the corresponding characteristic values of A, so 

A-i = 2 [(l/A(P>)x<P)'x(py J (8.49) 

Formally, this provides one method of inverting a matrix. But it may 
be more convenient in numerical work to invert the matrix by some 
other process, and then use (8.49) to determine the small characteristic 
values of A and their characteristic vectors. 


8.71, Iterative method for evaluation of characteristic values 
and characteristic vectors of a symmetrical matrix 

The characteristic value of greatest magnitude can be found as 
follows. Take an arbitrary vector X(o), with components X(Q)p of which 
the greatest in magnitude is unity. Form AX(o) and express it as a mul¬ 
tiple A(i) of a vector X(i) whose component of greatest value is unity. Then 
repeat the process with X(i) in place of X(o) to give a vector X(2) and so on. 
That is, form a sequence of numbers A(„,) and of vectors X(„,) so that 

where each A(,„) is chosen so that the component of X(„,) of greatest 
magnitude is unity. Then, unless the vector X(o) happens to have been 
taken orthogonal to the characteristic vector x<^) of the matrix, 

A(m) and X(„,) x<^) as m -> oo. 

The process thus far is also apphcable to non-symmetrical matrices. 

The rate at which successive values of A(„,) ultimately tend to their 
limit depends on |A<^YA<2)|^ and is greater the greater the value of this 

I G. Temple and W. G. Bickley, Rayleigh's Principle (Oxford, 1933). 
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ratio. If after the first few repetitions of the process of calculating 
A(^)X(„,) = AX(„j_ 2), the successive values of A(^) seem to be tending to a 
limit only slowly, the reason may be that ^ small. But it 

may be that X(o) happens to have been taken nearly orthogonal to x'^Y 
in case it has been so taken, it is as well to start the calculation again 
with another vector X(q) roughly orthogonal to the one previously used. 


Example-. To find x<i>, A<i> for the symmetrical matrix 


A = 


23 11 1 ]. 

11 -3 -2 

1-2 1 . 


(This is the matrix of the coefficients of the equations considered in the examples 
of §§ 8.22 and 8.3.) 

Starting with X((,) = (1,0,0) we have 

AX(o) = (-23. 11. 1) 

= -23(1, -0-48, -0 04), 

AX(i) = ( —28-32, 12-52, 1-92) 

= -28-32(1, -0-442, -0-068), 

AX{jj) = (-27-93, 12-462, 1-814) 

= —27-93(1, -0-4461, -0-0649), 

X(3) = (1, -0-4461, -0-0649), Ax^g, = (-27-971, 12-468, 1-827) 

= —27-971(1, -0-4458, -0-0653), 

X(4) = (1, -0-4458, -0-0653), AXj^, = (-27-9691, 12-4680, 1-8263) 

= -27-9691(1, —0-44579, -0-06530). 
Hence, to three decimals in A*** and five in x<i) 

Ad) = —27-969, xd) = (1, -0-44579, -0-06530). (8.50) 


X(o) = (1, 0, 0), 

X(i) = (1, -0-48, -0-04), 
X(3) = (1, —0-442, -0-068), 


Note: The number of figures in and X(„j) can be kept small in the early stages 
and increased as the approximation to Ad), xd) improves with repetition of the 
iterative process. 


After determining A^^ and x<^), the characteristic value A^-^ of next 
greatest modulus and the corresponding characteristic vector can be 
found by repeating the procedure used for determining A^^^ and x^^Y 
with the modification that X(q) and each X(^) is constrained to be ortho¬ 
gonal to x<^) before being multiplied b}’' A. That is, we form a sequence 
of numbers A(„,) and vectors X(„j) by the relation 

being a number determined by the condition that X(,„) should be 
orthogonal to x^Y \j^), as before, is determined so that the component 
of X(„j) of greatest magnitude is unity. 

If the work could be done exactly without rounding errors, then there 
would be no need to introduce the multiphers if X(o) were taken 
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exactly orthogonal to then each vector AX{^) would be orthogonal 
to The rounded values of AX(„_j) will, however, contain a small 
multiple of x^^^, and if this is not removed, it will give rise to an error 
which increases with further repetition of the iterative process, so that 
ultimately X(„) would tend to x^^^ and not to x^^). 

When x^^^ and x^^^ have been determined, the successive approxima¬ 
tions to x^®^ are similarly constrained to be orthogonal both to x^^^ and to 
x<2)^ and so on. 

This process has the advantage that there is no loss of significant 
figures as the work proceeds. On the other hand, since it makes use 
of the orthogonal property of the characteristic vectors, this property 
cannot be used as a check, whereas it forms a valuable check when the 
characteristic vectors are determined independently of one another. 
Also it is necessary to determine each characteristic vector accurately 
before starting to calculate the next, 

8.72. Richardson’s purification process for characteristic 

vectors 

In determining a characteristic vector x<^) and corresponding charac¬ 
teristic value by a method of successive approximation, it is an 
advantage to start with a trial approximation which is nearly orthogonal 
to any characteristic vectors about which something may be known from 
earlier stages of the calculation. One method of doing this has been 
outhned in § 8.71. Another process has been suggested by L. F. Richard¬ 
son, f and has two advantages; first, it does not require accurate deter¬ 
mination of other characteristic vectors, even those for which is 
greater than |A^5^| for the characteristic vector sought, and, secondly 
it enables the characteristic vectors to be calculated independently of 
one another so that the orthogonal property is available as a check. 

Any vector x (of n components) can be expressed as a linear combina¬ 
tion of the characteristic vectors x^p^ of A: 

X = x(i )+62 x(2)-(-...+6„ x^”) = 2 (8.51) 

p 

Multiplication by (A—ZI) gives 

(A-ZI)x = 2 (A(P>-Z)6p x^^’). (8.52) 

p 

If one of the characteristic values, say A^*"), were known exactly, and I 
were taken to be A^''^ the coefficient of the corresponding term in (8.52) 


f FhiL Trans. Roy. Soc.2^2 (1950), 439. 
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would be zero; that is to say, (A—A<’'>I)x is orthogonal to Similarly 
(A—A^’'>I)(A—A<*>I)x is orthogonal to x^*") and to x<*\ and so on. 

Even if A^''^ is not known exactly, the coefficient of x^*"^ in (8.52) will be 
relatively small provided that the value of I taken, say l^''\ is such that 
I^W—is substantially smaller than most of the other quantities 
|A(p)—ZW] for^ ^ r. Further, by repeating the multiplication by (A—ZI) 

we have (A-ZI)”‘x = ^ (A<P>-Z)”'6p x(p\ 

P 

so that starting from any arbitrary vector x, the vectors x, (A—Z^’'^I)x, 
(A—Z(’'d)2x,... are more and more nearly orthogonal to Similarly if 
are approximations to A<®\ and any vector x is multiplied by 
(A—Z<'')I)(A—Z^*^I), the result will be a vector nearly orthogonal to 
x^'") and to x<*^, and so on. 

If x^^^ is a characteristic vector to be determined, an arbitrary vector 
X can be described as a mixture of the required vector x<^^ with ‘impuri¬ 
ties’ in the form of multiples of the x^p^’s (p ^ q) in amounts represented 
by the coefficients in (8.51). The effect of multiplying by (A—Z^'’^!) 
or by (A—Z^’'T)”* can be described as ‘purification’ of the mixture by 
removal of most of the x^*"^ component from it, and similarly for the 
effect of multiplication by (A—Z^*^I)”*; this suggested the term ‘purifica¬ 
tion process’ used by Richardson. 

In order that such a purification process should be effective, the 
values used for quantities like Z<®> need only be approximations to 
characteristic values. This has two advantages in practical numerical 
work. First, for symmetrical matrices good approximations to charac¬ 
teristic values can be obtained, by use of formula (8.44), from approxima¬ 
tions to characteristic vectors which are only moderate; hence if only 
one characteristic vector is required, it is not necessary to determine 
the others to any great accuracy. And, secondly, it is possible to use 
simple rounded values for the Z’s if this would simplify the numerical 
work. 


Example: To detennine x<*>, x<®) for the matrix 


A = 

-23 

11 

r 


11 

-3 

-2 


1 

-2 

1 . 

without accurate determination of 




Suppose that in the example of §8.71 the successive approximation forx<^>,A*^> 
has only been taken as far as the second stage, with x^) = (1, —0-48, —0 04), and 
it is desired to find the other characteristic vectors and values. The best available 
approximation to A<^> at this stage is given by evaluating formula (8.44) for this 
vector X(i), and is —28-00. This, however, is so close to the actual value of A*^> for 
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this matrix that use of it would not illustrate how effective the purification process 
can be although IW is only a rough approximation to So in working the example 

here, the value = — 27 will be taken. Then 


A-iO)I = A+27I 


■ 4 11 r 

11 24-2 

.1-2 28 


This suggests x = (0,1,1) as a trial vector. For this vector x, 


(A + 27I)x = (12,22,26), 


for which the approximation 24(i, 1,1) is adequate at this stage. Hence wo start 
the approximation for the second characteristic vector with X(o) = i(l, 2,2). This 

AX(o)= i(l,l,-l). 


This differs considerably from x<o), so that although this X(o) is approximately 
orthogonal to it is far from (we shall see later that it is much more nearly 
in the direction of It is also some way from being orthogonal to the best 

approximation available to x<^>, namely (1,—0*48,—0*04), so that rather than 
using (1,1, — 1) as the next trial vector X(i), we repeat the purification process, 
which gives 

(A + 27I)(1,1,-1) = (14,37,-29) = 37(0-38,1,-0*78) 


and take x^) = (0-38,1, —0-78) as the next trial vector. This gives 


Ax<i) = (1-48,2-74, -2-40) = 2-74(0-54,1, -0-88) (8.53) 

and (X(i)Ax<i))/(X(()Ix<i)) = 2-95. (8.54) 

The further procedure depends on the results required. We will first consider the 
improvement of the approximation to x^^), and then the determination of x<^) 
without more information about x<2) than is expressed by (8.53) and (8.54). 

(a) Improvement of the approxhnation to x^^). Wc now have the approximations 
—28-0, +2-95, and the general result that the sum of the characteristic 

values is the sum of the diagonal elements of the matrix, in this case —25. Hence 
we can conclude that A^^^ is roughly 0-05, so that = 0 is a good approximation 
to A^^^ and multiplication by A itself will be effective in removing from a trial 
vector X any ‘impurities’ in the form of multiples of x^^>. Hence starting from 
(0-54,1, -0-88) given by (8.53) as the best approximation yet available to x<2>, we 
multiply by (A + 27I) to remove multiples of xt^) and by A to remove multiples 
of x<^) and also to examine the successive approximations of Ax to A^^^x. 

Since = 0 appears likely to be a better approximation to A^^^ than = 27 
is to A^^^ it will be best to carry out two or three multiplications by (A+27I) for 
each multiplication by A; and since the vector (0-54,1, —0-88) is already the result 
of a multiplication by A, and so contains only a small admixture of x<^>,let us start 
by some multiplications by (A+ 271): 


(A + 27I)(0-54, l,-0-88) = (12-28, 31-70,-26-10) 

= 31-70(0-387, 1,- 0-823), 

(A + 27l)(0-39, l,-0-82) = 29-93(0-392,1,-0-821), 

(A + 27I)(0-392, 1, -0-821) = 29-954(0-3922,1, -0-8211). 
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This has produced a vector nearly free from xO) but possibly still containing traces 
of To remove these we multiply by A; 

A(0-3922,1,-0-8211) = 2-9564(0-3918,1,-0-8216). 

The results of further multiplication by A alone diverge. This can be seen as 
follows. Let X be an approximation to xt^); 

X = x(2)-f 5^xO)-f tgX^^), 

where and 63 are small compared with 1 . Then 

Ax = A( 2 )[x( 2 ) + (AO)/A( 2 )) 6 iX( 1 ) + (A( 3 )/A( 2 )) 63 x( 3 )]. 

Now is greater than 1, and in this example it is about 10, so that any 

‘ impurity’ in x in the form of a multiple of x^^) is more prominent in Ax than in x. 
This building up ofx^^> can be avoided by further multiplications by — 

(A + 27I)(0-3918,1,-0-8216) = 29-953(0-39215,1,-0-82172), 

(A + 27I)(0-39215,1,-0-82172) = 29-9571(0-39212,1,-0-82171). 

This vector x = (0-39212,1, —0-82171) gives 

Ax = 2-9567(0-39217,1,-0-82171), 


(x'Ax)/(xTx) = 2-9568. 


So that to four decimals in A^^^ and a possible error of one or two in the fifth decimal 


in x^^) 


A^ 2 ) = 2-9568, x( 2 ) = (0-39212,1,-0-82171). 


(8.55) 


Notes: (i) The convergence would be much quicker if the value = — 28 instead 
of —27 were taken, but this working illustrates that the purification process is 
effective even if is only a rough approximation to A^^^ 

(ii) With a matrix whose elements are integral, as in this example, the numerical 
work is simplified if the Ts are taken to have integral values; in this example all 
elements of A and of (A —J(^)I) are numbers of two digits only, which makes the 
multiplications very quick. 

( 6 ) Determination of x(^), A^^h The most striking illustration of the power of the 
method is provided by the determination of x^^), the characteristic vector for the 
smallest value of |A|, without requiring that x^^^ or x^^) should be determined more 
accurately than they are at the stage of the calculation reached at formula (8.53) 
and (8.54); comparison of formula (8.53) with the result (8.55) shows that the 
approximation to x^^) that stage is decidedly rough. 

We have used = — 27 as an approximation to A^^^ and can now adopt = + 3 
as an approximation to A^^^; so repeated multiplication of an arbitrary vector x by 
( A4*27I)(A— 31) will produce a sequence of vectors whose directions will converge 
to that of x^^). Now 


(A4-27I)(A-3I) = 

' 4 

11 

1 ‘ 

"-26 


11 

24 

-2 

11 


1 

-2 

28. 

1 


= — 

-18 

24 

20 ' 

= 


24 

19 

33 



20 

33 

51. 



11 r 
-6 -2 

-2 -2 

B (say), 


and we have seen that x = J(l, 2 , 2 ) is nearly orthogonal to xd)and differs consider- 
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ably from x**), so let us take it as a first approximation to x*®). Then successive 
multiplications by B give: 

= ^(35,64,94) = -94(0*37,0-68,1), 
B(0*37,0*68,1) = -(29*66, 54-80, 80*84) 

= -80-84(0-367,0*678,1), 

B(0*367, 0*678,1) = -80*711(0*3675,0*6776,1), 

B(0-3675,0-6776,1) = -80-7108(0-36733,0*67766,1), 

B(0‘36733,0-67766,1) = -80-7094(0-36739,0-67763,1), 

B(0*36739, 0*67763,1) = —80*7096(0-36737,0*67764,1), 

B(0*36737, 0*67764,1) = -80*7095(0*36738,0*67764,1). 

This process could be continued indefinitely, to give as many decimals in as 
might be required. With a possible error of 1 in the fifth decimal 

xt3) = (0-36738,0-67764,1) 

and with this value of 


(x( 3 )'Ax< 3 ))/(x< 3 )Tx< 3 )) = 0*012093, 

which is therefore an approximation to 
The collected results for this matrix A are 


= -27*969, 
A<2) := 2*9568, 
A(3) = 0*012093, 


x(i) = (1, -0*44579, -0*06530)^ 
x(2) = (0*39212,1, -0*82171) , 

x(3) = (0-36738,0-67764,1) j 


(8.56) 


A(i),x^i)being given by (8.50) and A^2)^x<2)by (8.55). The best check on these results 
is provided by verifying that the relations between characteristic vectors and 
between characteristic values are satisfied. 

Since the characteristic vectors have been determined independently of one 
another, they can be checked by verifying that the orthogonality relations are 
satisfied. The results (8.56) give 

= - 0 * 00001 , ^xfxf = - 0 - 00001 , 

J^xfxf^ = -O-OOOOOg, 

which differ from zero by amounts within the tolerance for rounding errors. 

The sum of the characteristic values should be equal to the sum of the diagonal 
elements, which is —25 in this case, and this relation is satisfied exactly .tp three 
decimals. And since the elements of the matrix are integral, their determinant is 
integral, so the product of the characteristic values should be integral. F or the results 
(8.56), A^^W^W^^= —1*00007, which differs from —1 by an amoirnt within the 
tolerance for rounding errors. 

A further check is provided by building up the matrix A from its characteristic 
vectors and characteristic values according to formula (8.48); this, however, does 
not give a good check on x<^), since its contribution is multiplied by the small 
factor A^^h 

Notes: (i) The ratio |A^^VA^^^| of the greatest and least in magnitude of the charac¬ 
teristic values is about 2300, and the large size of this quantity is an indication 
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of the degree to which the equations Ax = b, with this matrix A, are ill-con¬ 
ditioned. This character of A has not, however, introduced any difficulty in finding 
the characteristic values. 

(ii) If the characteristic values and vectors are found in succession, the last (in 
this case the third) can be found as follows. 

From an arbitrary vector x, a vector X, orthogonal to xO) and x^^^ is formed by 

X = (x-^iX(1)-/X2x(2)) (8.57) 

and determining appropriately. Since is orthogonal to it follows that 
the required values of /Xi,/X 2 are 




(8.58) 


X should then be multiplied by such a factor that its greatest component is 1, and 
the process repeated, as a check and to remove effects of rounding errors as far as 
possible. 

If the elements of A are integral, the following alternative process can sometimes 
be used to determine As soon as and A^^^ are found to the accuracy 

A<i) = -27*969±0-001, A^2) = 2-957±0'001, 

it follows, from the relation 

A(1)-[-A^^^+A^^^ = sum of diagonal elements = —25, 
that A^^Mies in the range A<^^ = 0*012 ±0-002, 

and hence = -0-99±0-17. 

But since the elements of A are integral, this product must be integral, so must 
be —1. Hence ^ _l/A(i)A(2)= +0-01209. 

(iii) The purification process can be used to hasten the convergence of the 
approximation to A^^^ x<^). From the result (8.54) it follows that multiplication 
by (A—31) will be effective in removing multiples of x^^^ from a trial approximation 
to x(^). Hence after the second step in the example as worked in § 8.71 we could 
proceed as follows: 

(A-3I)(1,-0-442,-0-068) - -30-930(1,-0-44578,-0-06531), 

A(l, -0-44578, -0-06531) = -27-9689(1, -0-44578, -0-06530), 
reaching the result in two fewer steps than in § 8.61. 

This example illustrates some possibilities of Richardson’s purifica¬ 
tion procedure; for developments of the idea, and other examples, 
reference should be made to Richardson’s paper.f 

f For some other methods, see C. Lanezos, Journ. of Researchy Nat, Bureau of Standards^ 
45 (1950), 255; J. H. Wilkinson, Proc, Camb, Phil, Soc. 50 (1954), 536, and Proceedings 
of a Symposium on Automatic Digital Camputation ^ N.P.L. 1953 (H.M.S.O. 1954), 
ch. 18; R. A. Brooker and F. H. Sumner, Proc, Inst, Elect, Eng. 103 , Part B (1956), 
Supplement No. 1, 114. 
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8.73. Relaxation process for characteristic vectors 

Another method of determining characteristic vectors is a form of 
relaxation process. 

If^ is any given number and we try to find by a relaxation process a 
solution of the equations 

«21^1+(«22 —^)^2+«23^3+-" = ^ CtC. 

in which not all the x^’s are zero, then unless I is a, characteristic value, 
it will not be possible to reduce all residuals to zero. But if the x^’s are 
components of a characteristic vector, corresponding to a characteristic 
value A, then 

(Uii— Z)a:i+ai2a:2+«i3»^3+-” = 

® 21 ^ 1 “ 1“(®22 ^)^ 2 ~ I ~®23 ^ 3 “ I “ •" “ ^)^ 2 > 


(8.59) 


so that the ratio [(residual of the jth equationwill be the same for 
all values of j. 

If therefore we fix one of the ar^’s at a non-zero value, and in the 
relaxation process aim not at reducing all the residuals to zero but at 
making the residual of each jth equation proportional to Xp then a 
change of I, forming an improvement in the approximation to A, will 
have the effect of reducing all the residuals simultaneously. 

If an approximate characteristic vector x can be estimated on inspec¬ 
tion of the equation, then an approximation A to the corresponding 
characteristic value is given by formula (8.44), and this can be taken as 
the best available value, at this stage of the calculation, to 1. 


Example: Consider again the matrix used in the examples in the previous sections, 
namely ^ _ 


-23 

11 

1 


11 n 

-3 -2 

-2 1 


Inspection suggests that x = (2, — 1,0) is an approximation to a characteristic 
vector. For this x, the vector Ax is 

Ax = (-67,25,2) 

and 2 xj AjkxJl = - 139/5 = - 27-8. 

jk / j 

With I = —27-8 the matrix of the coefficients of equations (8.53) is 

4-8 11 1 ■ 

11 24-8 -2 

1 -2 28-8 


A-ZI = 
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and the beginning of the relaxation process, keeping fixed, is as follows: 


Operations 

table 

o o 

X2 

1 

0 

^3 

0 

1 

Ri 

11 

1 

i?2 

24-8 

-2 

i?3 

-2 

28-8 


Xi 

^2 

^3 

Ri 

i?2 

Rz 

Relaxation 

2 

-1 

0 

-1-4 

-2-8 

4-0 

table 



-0-14 

-1-54 

-2-52 

-0-032 



01 


-0-44 

-0-04 

-0-232 




0-01 

-0-43 

-0-06 

0-056 



001 


-0-32 

0-188 

0-036 


2 

-0-89 

-0*13 

-0-32 

0-188 

0-036 Checked 


Although the residuals are not very closely proportional to the values of the or^’s, 
they are now of such signs and magnitudes that a change of I which would reduce 
Ri to zero would also decrease |i? 2 | and \Rs \ considerably. 

When a stage such as this has been reached, it is best to calculate a new approxi¬ 
mation to A; in this case, with 

X = (2, -0-89, -0*13), 

Ax = (-55-92,24-93,3-65), 

= 134-50/4*8090 = -27*968 

jk I j 

and with this value of I, the residuals, to three decimals, are 

0*016, 0*039, 0*014; 

as expected, these values are substantially smaller than those at the end of the 
relaxation table above. A further relaxation ean then be earried out starting from 
these values. 

An alternative way of evaluating an improved approximation to A 
is as follows. When residuals have been obtained such that R^ is 
roughly proportional to Xj, then x is an approximation to a characteristic 
vector of the matrix (A—ZI), and a better approximation to the corre¬ 
sponding characteristic value (A—Z) can be made by use of the formula 
(8.44) applied to the matrix (A—ZI). Now the vector R formed by the 
residuals Rj. is given by (A—ZI)x = R, so an approximation to A—Z is 

} ' } 

whence an improved approximation to A is given by 

A = Z+ 2 RJ 2 (^D- (8.60) 

A similar process can be carried out for the other characteristic vectors 
in order of decreasing |A|. For these, however, it is necessary either to 
eliminate each one from the matrix as it is calculated, or to ensure that 
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each as it is calculated is made orthogonal to all those already deter¬ 
mined, as in § 8.71. 

Characteristic values of linear ordinary differential equations with 
two-point boundary conditions can be obtained by a combination of 
this technique with the replacement of derivatives by finite differences 
as in § 8.6.f 

t See L. Fox, Proc, Camb, Phil. Soc. 45 (1948), 60, § 8. 
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NON-LINEAR ALGEBRAIC EQUATIONS 

9.1. Solution of algebraic equations 

By an ‘algebraic’ equation is meant, in this chapter, an equation 
f[x) = 0 not involving derivatives or integrals of f{x), and of which a 
solution is a number, as distinct from a differential equation of which a 
solution is a function of the continuous variable x. It does not imply 
that the function/(a:) whose zeros are to be found is an algebraic function. 
For example the equations 

a^-[-5x- — 3x —2 = 0 
and e^sinic = 1 

are both ‘algebraic’ in this sense. 

f{x) being a given function of x, the problem of finding the roots of 
f{x) = 0 is often best dealt with in two steps, the first concerned with 
locating the roots roughly, to two or three significant figures, and the 
second with improving these rough values. 

The solution of an algebraic equation can be regarded as a process of 
inverse interpolation, for iif{x) is tabulated as a function of x, then the 
determination of the value of x for which f{x) has any given value, of 
which zero is a special case, is just the situation with which inverse 
interpolation is concerned. Once a solution has been located approxi¬ 
mately, tabulation of the function in the neighbourhood of that solution, 
followed by a process of inverse interpolation, is one way of determining 
it more exactly. Another method is to use an iterative process; this is 
considered in § 9.3. 

9.2. Graphical methods 

Use of graphs is often a valuable method of locating approximately 
the roots of an equation f{x) = 0. Either the function f{x) itself may 
be graphed and its intersections with the a:-axis determined, or the 
equation may be written in the form ffx) = f^i^), and its roots deter¬ 
mined by the intersection of the graphs of y = fi{x) and y — f^ix)] it 
may be possible to avoid some calculation by this process. In some cases 
graphs using some argument other than x, such as log a: or 1 jx, may be 
useful. The best procedure will depend on the form of the function 
f{x), and it is difficult to lay down any general rules. 
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The following examples give suggestions for handling some kinds of 
equations: 

(i) Equations of the form f(x) s x^-\-ax-\-b = 0. Use graphs of = x" and 
y = —{ax+h). 

(ii) Equations of the form 

f{x) = x’*+OiX”-i-f a„ = 0. 

Evaluate/(x) directly (see §3.2) or by building up from its differences (see §4.42) 
for, say, x = — 5(0-2)-|-5; for |x| > 4, say, take y = Ijx and evaluate 

F(y) = an2/"+an-i2/"“^+ —+Oi2/+«o- 
Use graphs of/(x) against x and F(y) against y. 



(iii) e^^sinx = 1. Write this as sinx = e~i®, and determine the intersections 
of y = with y = sinx (see Fig. 13). This avoids calculation of the products 
ei^sinx. 


9.3. Iterative processes 

By an iterative process is meant one in which the equation f{x) = 0 
is expressed in the form ^ 

and we try to find a solution by constructing a sequence by the 
relation 


^n+l 


(9.1) 


If, to the degree of numerical accuracy to which the work is carried out, 
= x^, then this value of is a solution of the equation to that 
degree of accuracy. 

Let a; = X be the solution of the equation, and let 




(9.2) 
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SO that is the error in regarded as a solution of the equation. An 
important feature of an iterative method is the way in which this error 
varies with the number n of repetitions of the iterative process. This can 
be examined by expanding the right-hand side of (9.1) in a Taylor series. 
Then, since x = 'X. satisfies x = F(x), it follows that 

In-H = ®i^n+®2^n+®3ln + ---> (^•^) 

where = F^^\X)jk\. 

If ^ 0, then the errors of results of successive repetitions of the 
iterative process are ultimately related by 

^n+l ^n+m ' ®1 ^n> 

in order that the process should converge, |ai| = |i^'(A’)| must be 
less than 1, and the magnitude of the error then decreases exponentially 
with n increasing. This means that the number of additional correct 
significant figures obtained from each repetition of such a process (or; 
more often, the number of repetitions required to obtain each new 
correct significant figure) is the same, however many figures have been 
obtained. Such a process is called ‘first-order’. 

But if «! = 0, Ug 7 ^ 9 in (9.3), then the successive errors are ulti¬ 
mately related by 

^n-H = « 2 ^n. whence a^^n+m= 

where = \F"{X). The number of correct significant figures is approxi¬ 
mately doubled for each repetition of the iterative process, so that the 
better the approximation of x^toX, the easier it is to improve it further. 
Such a process is called ‘second-order’, and once a fair approximation 
to a; = Jl has been attained, a second-order process is very greatly to 
be preferred to a first-order one; but it must be started from an approxi¬ 
mation good enough to ensure that lug^ol < 1- It will be shown in 
§ 9.32 that from any first-order process it is possible to derive a second- 
order process. 

If Uj = 0, = 0, Ug 7 ^ 0 in (9.3), then the successive errors are 

ultimately related by 

= whence 

such a process is called ‘third-order’. The formula for a third-order 
process is usually more comphcated than that for a second-order process 
for the same equation, and the convergence of a second-order process 
is already so fast once a good approximation has been obtained that the 
advantage of still quicker convergence obtainable from a third-order 
process may be more than offset by the more comphcated formulae 
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which have to be evaluated for each repetition of the iterative process, 
and third-order processes are not much used in practice. Second-order 
processes, however, are widely used. 


9.31. Examples of iterative processes 

(a) Newton's process for a square root 

An important example of a second-order process is one for a square 
root, usually known as ‘Newton’s process’. If b is the number whose 
square root is required, this process consists of forming the sequence 
W defined by = iK+(6K)]. 

For this process, X = b^, and F{x) = ^[x-\-{bjx)], giving 
F'{x) = ^(i-bjx^), F'{X) = 0, 


F"(x) = blx\ F"{X) = \[X, 

so that dj = 0, 0 in (9.3), and the process is second order. 

As an example of the appUcation of this process, consider the evalua¬ 
tion of Vl2, starting from x^ = 2. 


*0 — *1 — 

*1 = 4, = i(4+l5) = | = 3-5, 

^2 — h ^3 — ~ ~ 3*4643, 

x^ = 3*4643, 12/a;3 = 3*46390,32, x^ = 3*46410,16, 

x^ = 3*46410,16, Ujx^ = 3*46410,16303, x^ = 3*46410,16151. 


Notes: (i) For the first two or three iterations it may be easiest to work with the 
numbers in the form of rational fractions; later it is more convenient to work 
with them in decimal form. 

(ii) In this example, an unnecessarily bad approximation has been taken as a 
starting-point to illustrate the convergence of the process from even a very poor 
value of Xj. A more important application is the improvement of an already fairly 
good approximation such as Xj. From Barlow’s Tables a square root correct to 
four figures can always be obtained without any interpolation; then one application 
of Newton’s process will give eight figures, and another will give fifteen figures at 
least. 


Newton’s process is not the only second-order one for a square root: 
another is given by 

^n+i = xJZb—xl)l'2b = x^[\ + {b-xl)l2b]. (9.4) 

This process does not converge as fast as Newton’s, since for a given 
value of 6, the value of in the series (9.3) is greater. But it has one 
feature which may be an advantage, namely that the divisor in (9.4) is 
constant instead of being different at each stage of the iteration process 
as it is in Newton’s process. 
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( 6 ) The Newton-Raphson process 

A general second-order process for the solution of/(a:) = 0, at a point 
not in the neighbourhood of a maximum or minimum of f{x), is one 
given by ^ (9.5) 


ji+i 




This is known as the Newton-Raphson process; Newton’s process for 
a square root is the special case of it for the function/(x) = x^—b, and 
(9.4) is the special case of it for/(x) = l—bjx^. For the general Newton- 
Raphson process, 

F{x) = x-f{x)ir{x), F'{x) = -f{x)r{x)i[f\x)f, 
and since/(X) = 0, it follows that F'{X) = 0, so the process is second 
order. 

Expressed in terms of the graph of/(x), the Newton-Raphson process 
is equivalent to linear interpolation along the tangent to the curve 
y = at x^ (see Fig. 14). 

(c) The 'rule of false position' 

Another iterative process is equivalent to linear interpolation along the chord 
joining [xo,f(Xo)] to [Xn,f{x„)] (see Fig. 15). This gives 

^n+l = ^0-iXn-^o)foKfn-fo) = (Xofn-X„fi,)l{fn-fo)- 
This method, however, is only first order, though if Xq is a fair approximation to 
X, the coefficient in (9.3) is small; successive errors are related by 

in+i = U^o-X)r(X)Uf'(X), 

approximately. It has the advantage that it does not require the evaluation of f '(x). 

A disadvantage of methods such as the Newton-Raphson and the 
method of false position is that they involve the evaluation of/(x) and 
f'{x) at a number of values of x which, though systematic in the sense 
that each is calculated from the previous one by the same formula such 
as (9.6), are irregularly spaced, and such a set of numbers is difficult to 
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check adequately. An advantage is that a mistake in an intermediate 
value of Xu does not affect the final result; it is just equivalent to starting 
a new iteration with this erroneous value of as But this does not 
eliminate the possibility of a mistake in the last repetition of the iterative 
process. Tabulation of f{x) at equal intervals of x followed by a process 
of inverse interpolation is a process which provides more, and simpler, 
checks against occasional mistakes. 


Example: To find the root of xtanx = ^ which lies between x = 0'6 and 0'7. 
(a) By the Newton—Raphson process 

There are several forms in which this equation can be written, for example: 
/(.r) = xtanx—J = 0; f(x) — 2a:—cotx = 0; 
f(x) = 2a: sin a: — eosa; = 0. 


The third of these will be adopted, as it gives the most convenient formula for 
f'(x), namely ^ 2a:cosx + 3sina:. 

Starting with Xq = 0*6, sinxg = 0*5646, cosXq = 0*8253, wc have 

/(xq) = 2xosinxo — cosxq = —0-1478 | /(*^o)//'(-^o) = —0*0551, 

/'(Xq) = 2xoCOSXQ+3sina:o = +2-6842 r x^ = 0-6 + 0-0551 

= 0-6551, 


Xy = 0*655, sinxy = 0-609159 cosxy = 0*793048 

f(Xi) = 2xi sinXj — cos= +0*004950 | = +0*001727, 

f'(xi) = 2xiCOSXi + 3sinxi = 2*86637 Xo = 0*655-0*001727 

= 0*653273, 


Xg — 0*653273, sinxg = 0*607788, cosxg = 0*794099 

f {^ 2 ) ~ 2x2sinx2 — COSX2 = +0*00000771 f (^ 2 ) lf '(^ 2 ) — +0*0000027, 

/'(X 2 ) — 2x2eosx2 + 3sinx2 = 2*86097 ; Xg — 0-653270. 

Notes: (i) The first approximation Xq = 0-6 is a rough one and four-figure values 
of sinx, cosx are adequate at this stage; more figures are u.sed later when the 
accuracy of x^^ has been improved. 

(ii) For the second stage of the iteration, Xj is taken as 0-655 instead of the value 
0*6551 obtained from the first stage. It is not to be expected that the fourth decimal 
of this value will be correct, and the rounded value Xy — 0*655 enables tables with 
interval 8x = 0*001 f to be used without interpolation. For the third stage, how¬ 
ever, interpolation in the tables is necessary. 

(iii) For the third stage (and later stages, if any) it would be adequate to use 
f'{xy) instead of recalculating/'(x^) for each new value of x„. This makes the 
method formally only first order, but the coefficient in (9.3) is so small in such 
a case that the convergence of the first-order process is adequate for practical work. 


(6) By inverse interpolation 

To solve the equation by inverse interpolation, it is most convenient to take it 
in the form ^ 2x-cotx = 0 


I Chambers's 6-Figure Tables, voh 2 (1949), for example. 
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as this avoids the calculation of any products and involves the least reference to 
tables. Evaluation of J{x) to two or three decimals at intervals 0*05 or 0-02 in the 
range x = 0-6 to 0-7, as might be used for a rough plot, locates the root as lying 
between x = 0-65 and 0*66, and taking 0-01 intervals we have the following table: 



cotx 

/(x) = 2x—cotx 

8*/ 

0-64 

1-343104 

-0-063104 

47668 


0-65 

1-315436 

-0-015436 

46950 

-718 

0-66 

1-288486 

+ 0-031514 

46265 

-685 

0-67 

1-262221 

+ 0-077779 



Inverse interpolation for/(x) = 0 at a: — 0*65+jo(0*01) gives 

^ ^ 1 6 4 3 6 _ .i4^L3_ 

P — 46960 ^ [P) • 50 

= 0*32878-(0*02988), 

and iterative solution of this equation gives p = 0*32715, x = 0*6532715. 

Alternatively, 0*002 or 0*001 intervals could be taken between x = 0*65 and 
0*66; for example 


0-650 

cotx 

1-315436 

f{x) = 2x—cotx 
— 0-015436 


0-652 

1-309989 

9447 

-0-005989 

— 29 

0-654 

1-304571 

9418 

+ 0-003429 

— 29 

0-656 

1-299182 

9389 

+ 0-012818 



and linear interpolation is now adequate to give five decimals in x. 

Note: The two methods may be combined; for example after obtaining the approxi¬ 
mation x^ by the Newton-Raphson method, the approximation to the root may be 
improved by tabulation at 0*001 intervals in the neighbourhood of x = 0*655, 
followed by inverse interpolation. This avoids the interpolation in tables which 
has to be done if the Newton-Raphson process is continued. 


9.32. Derivation of a second-order process from a first-order 
process 

If it is known that an iterative process is first order, this knowledge 
enables a better approximation to the solution to be obtained by an 
application of the process of 'exponential extrapolation’ (see § 3.4(a)). 
If the first term in the expansion (9.3) were the only term, then we would 

Uii = UU = “i. that is, 
exactly, so that X would be given by 


(^2—X)(a:o—X) = (xj—X)2. 

Unless the higher terms in (9.3) are negligible, this will not give exactly 
the value of X, but an approximation, say X*, to it: 


X* = 


ry ry _/y.2 

*^2 *^0 *^1 


Xo 




X 2 2xj -j" ^0 


(9.6) 
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(see eq. 3.12), which will usually be a substantially better approximation 
than X 2 . We can then repeat the process starting with X(, == X*. 

In general, let be the result of this process, starting with x^ — X*. 
Then it can be shown that the process of forming the successive values 
of X* is second order.f 

Example'. To solve the equation a:*—6a:-|-2 = 0 by writing it in the form 

X = 6—(2/a:) 

and using an iterative process. 

In this case the function F(x) of § 9.3 is F{x) — 6—(2/a:), F'(x) = 2/x* and 
though the solution a: = X is not yet known, we can be sme that F'(x) is not zero 
there, hence the iterative process = 6 —(2/a:„) is first order. 

Starting with X* = 3 we have 


Xq 3, a:^ — x^ — 

Then, from formula (9.6), 


Then with x^ 


x, — x, = 


^ 8 


x^ — 2x^-j-Xo 
(7/24)* 17 

—49/24 “ 3 “ 


49 

249 


5-66667. 


X* = we have 


x^ = 6-2.3/17 = 5-64706, 
then from formula (9.6) again, 

X 2 —Xi = — 0 - 00122 ; 


X} = 


5-64584— 


( 0 - 00122 )* 

0 - 018 ^ 


Xj = 6-2/5-64706 = 5-64584; 


X2—2x^-l-x„ — 0-01839; 

= 5-64584-0-00008 = 5-64576, 


which is only in error by 1 In the fifth decimal place. 


9.4. Multiple roots and neighbouring roots 

Particular care is necessary when the eoeffieients in the equation are 
in the neighbourhood of values for whieh the equation has multiple 
roots. The values of the roots are then particularly sensitive to the 
values of the eoeffieients and to rounding errors. To take a simple 
example, the equation 

16-00 = 0 has a double root x = 4, 

16-01 = 0 has no real root, 
a:2—8a:+15-99 = 0 has roots 3-9, 4-1, 
so that a change of less than one part in 1000 in the constant term affects 
the roots by one part in 40. The situation is clear here, but may not be 
when the equation has other roots or involves transcendental functions. 
If a repeated root is suspected, either from the results of this process or 
from the graph, then careful numerical evaluation of the function should 
be carried out in the neighbourhood .of the suspected repeated root, 
f S©© D. R. Hartree, Proc, Camb, Phil, Soc, 45 (1948), 230. 
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Two (or more) close but not equal roots may be more troublesome than 
a true repeated root. If two close roots are suspected from examination 
of a graph or on other evidence, there will certainly be a root of/'(a:) in 
this neighbourhood, say x = and it is best to determine this first, 
and to evaluate/(a„,). If this has the same sign as/(a) at neighbouring 
values of a, then there is no real root; if/(a„,) has the opposite sign to 
/(a) at neighbouring values of a, then there are two real roots. Since 
f'{x^) = 0, Taylor’s series for/(a) in the neighbourhood of x^^ begins 

fix) =/(aJ+i(a—aJ2/"(aJ, 

and if (a—a„,) is sufficiently small (and/"(a„j) is not too small) 

x—x^ = ±[2{/(a)-/(aJ}//"(aJ]i. (9.7) 

This, if not already accurate enough, will provide starting values for a 
further approximation to these roots. 

The calculated values of these roots will be ver 3 '^ sensitive to rounding 
errors; if/(a) is given by a formula which can be evaluated to any degree 
of numerical accuracy (for example, by a polynomial), the roots can 
be evaluated to any accuracy required; but if the evaluation of /(a) 
involves reference to tables, the accuracy of the calculated values of the 
roots may be small. 

Example'. To find the smallest positive root of xcos^a = 0'4115. 

A graph shows that f{x) = x cos^a has a maximum of about 0-41 in the neighbour¬ 
hood of a = 0-65. 

For this funetion, f'{x) — — eosx(2xsinx — eosx), and this has a zero at 
Xjn = 0-65327 (see § 9.31, example), where/(a) = 0-411949; also 

= — cosa„,(2a„,eosa„,H-3sina„j) = —2-272. 

So /(a)-/(aj = 0-4115-0-411949 = -0-000449, 

and the smallest root of aeos^a = 0-4115 is approximately 

0-65327—[2( — 0-000449)/( — 2-272)]i = 0-65327 — 0-01988 

= 0-63339. 

Note: The fifth deeimal of this value of a is not determined to i 1 by the sixth 
deeimal of/(a) or/(a„,). 

9.5. Special processes for special types of equations 

The methods so far considered have been general methods applicable 
to any kind of algebraic equation, in the sense explained in § 9.1. For 
some special kinds of equations f{x) = 0, and particularly for those in 
which/(a) is a polynomial in x, there are special methods. 

Polynomial equations, like linear simultaneous equations, are met in 
contexts of two kinds. In one the coefficients are all known exactly and 
are usually integral; in the other they are only known to within a certain 
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tolerance because they are results either of observations subject to 
experimental errors or of other calculations which are subject to rounding 
errors. 

In the latter case it may be important to know the range of uncertainty 
of the solution arising from the tolerance in one or more of the coefficients. 
If the polynomial is 

f(x) = aoa;”+aiJ:«-i+a2a:»-2+...4-a„_iX+tt,, = 0 (9.8) 

consider the first-order variation of a root x = X with one of the co¬ 
efficients, If the root X changes by AX when changes by Aa;^., then 

f'{X) ^X^X-^ Aa,. = 0, 
that is, AX = -[X”-''//'(X)] ^a^, 

and for changes (not necessarily equal) in all the coefficients 

AX = -[| (X"-*^-Aaj]//'(X). (9.9) 

This shows that the roots are particularly sensitive to the values of the 
coefficients in the neighbourhood of a stationary value of the function. 


9.51. Quadratic equations 

The roots of a quadratic equation ax^-\-bx-\-c = 0 can be evaluated 
from the standard formula 


X = [—4ac)^J/2a, (9.10) 

and this is probably the best way of evaluating complex roots. How¬ 
ever, as already pointed out (§ 3.4 (6)), this is not always the best way of 
determining numerical values of the roots when they are real, and par¬ 
ticularly not if the ratio of the roots is large (b^ ^ 4ac). 

A better practical method in many cases is an iterative process based 
on use of the relations 


Xi-^X 2 = —bja, x^x^ = c/tt, 

where x^ and x^ are the two roots. If x^ is the root of greater modulus, 
then successive approximations to the roots can be evaluated by using 

the formulae = -{bja)-X 2 , X 2 ^(cla)lx^ (9.11) 

alternately, starting from the approximation ^2 — 0 if no other is easily 
available. 

This process, though only first order, can be carried out so easily, and 
when b^ ^ 4ac it converges so quickly, that it is unnecessary to refine it 
further. If b^ is not considerably larger than 4ac, it may be convenient 
to use a second-order process derived from this first-order process as 
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explained in § 9.32, Elimination of between the two equations (9.11) 

x^ = -{bla)-{cja)lx^, 

which is the general expression of which the example given in § 9.32 is 
a special case. 

9.52. Cubic and quartic equations 

A cubic equation with real coefficients has at least one real root, say x^; if it is 
determined, division of the cubic by (x—Xi) gives a quadratic which can then be 
solved by the standard formula or by iteration. For determination of the real root 
or roots the general methods of the previous sections will often be best. 

There are also special methods available which depend on reducing the cubic 
to a standard form. If the cubic is 

ax^-\-bx^-\-cx~^d = 0 

then the substitution y = jS(a;4-6/3a) reduces it to a cubic in y without a y^ term; 
then either the coefficient of the term in y is zero, or a real value of jS can be chosen 
so that the ratio of the coefficients of the terms in y and in y^ is either +1 or — 1. 
Thus any cubic can be reduced to one of the forms 

y^+D = 0 , y^ + y + D = 0 , y^—y + D = 0 ; 

the solution to an equation of the first of these forms can be found directly from a 
table of cube roots, and tables of the roots of the equations of the other two forms 
have been evaluated (for details, see the Index of Mathematical Tables), 

If in the substitution y — jS(x4-6/3a), jS is chosen so as to reduce the equation 
to the form 4.y^±Zy-D = 0 

the further substitution y — sinh u (if the sign of the middle term is -f), ?/ = cosh u 
(if the sign of the middle term is —, and D > \) ot y — cosu (if the sign of the 
middle term is —, and Z) < 1) can be used to reduce it further to 

sinh Su = D, cosh 3u = D, or cos 3u = D 

respectively and the solution found from tables of hyperbolic or circular functions. 
For a quartic equation 

ax^~^bx^~\-cx^~\-dx-\-e = 0, 

the cubic term can be removed by the substitution 2 / = (x-\-b/4a). Let the resulting 
equation be yt^Cy’^^jyy^E = 0 

and let {y^^oiy-]-^), (2/^+a2/+y) be quadratic factors of the left-hand side. Then, 
multiplying these factors and equating coefficients, 

^+y-cc^ = C, (9.12) 

a(iS-y) = A (9.13) 

jSy = E, (9.14) 

Elimination of P and y gives a cubic equation for a, but instead of carrying out 
this elimination algebraically, it is more convenient to proceed as follows. From 
equations (9.12), (9.13) 

jS+y = C + = D/oc, (9.15) 

From these, )9 and y can be evaluated for a set of values of a, and the value of a 
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for which 4jSy = 4lE in agreement with (9.14) can be determined by trial and 
inverse interpolation. If the roots of the quartic are not all real, this value of a 
is unique; if the roots are all real, there are three values of a corresponding to the 
three ways in which the real linear factors of the left-hand side can be separated 
into two pairs. 

Example: Find the roots of the equation 

6-4x3+19-8x2—31-5ic + 25 ^ q 
T he substitution oft/ = x —1-6 gives the equation for y 
2/i+4-442/*-0-9082/+6-6272 = 0, 
so equations (9.15), (9.14) become 


4*44+a2, jS—y = - 

first trial set of values of a gives 

-0-908/a, 

4j3y = 22-5088. 

a 

^+y 

^-y 

2^ 

2y 

4^y 

0-2 

4-48 

-4-64 

— 0-06 

-J-9-02 

-0*54 

0-3 

4-53 

— 3-027 

-1-1-503 

7-557 

+ 11*36 

0-4 

4-60 

— 2-27 

2-33 

6-87 

16*01 

0-5 

4-69 

J 

-1-816 

2-874 

6-506 

18*70 

0-6 

4-80 

— 1-513 

3-287 

6-313 

20*75 

0-7 

4-93 

-1-297 

3-633 

6-227 

22*62 

0-8 

5-08 

-1-135 

3-945 

6-216 

24*52 

lese values indicate that to give 4j3y = 22-5088, the value of a 
id further trial values in this neighbourhood give 

is about 0- 

0-693 

4-9202 

-1-3102 

3-6100 

6-2304 

22-492 

0-694 

4-9216 

-1-3084 

3-6132 

6-2300 

22-510 

0-695 

4-9230 

-1-3065 

3-6165 

6-2295 

22-529 


18 

19 


from which a = 0-69395; f^is value can be improved if required by taking the 
approximation farther. The evaluation of the roots of the equation for y (which 
in this example are all complex) from formula (9.10) is straightforward, and from 
these the roots of the original equation for x follow directly. 


9.53. Polynomial equations 

Por solving polynomial equations there are available a number of 
special methods.f Various theorems in the theory of equations can be 
used to determine how many roots lie in various ranges of x, and various 
special methods are available such as Horner’s method or a method 
known as 'root-squaring’J which depends on forming an equation whose 
roots are some high power of the roots of the equation to be solved. 

The root-squaring method, however, only gives the magnitudes of 
the roots and not their signs, and some evaluation of/(x) is necessary in 
order to determine their signs. Also both methods, as usually presented, 


t For a survey, with particular reference to equations of high degree, see F. W. J. 
Olver, Phil. Trans. Roy. Soc. 244 (1952), 385. 

t See E. T. Whittaker and G. Robinson, Calculus of observations (Biaokie, 4th ed., 
1944), § 64. 
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are deficient or lacking in current checks, and their results should always 
be verified either by substitution in f{x) or by evaluating f{x) at a 
numerically convenient set of values in the neighbourhood of each root 
followed by interpolation. The root-squaring method in particular 
offers too many opportunities for mistakes for any alleged root to be 
accepted without such investigation. 

Some evaluation of f{x) is therefore required in any case, and it seems 
better for most practical purposes to use the general methods already 
considered, namely the use of graphs for approximate location of the 
roots followed either by evaluation in the neighbourhood of each root 
and inverse interpolation, or by an iterative process,! rather than to 
use the special methods available for polynomial equations. 

9.54. Repeated roots 

Repeated roots of polynomial equations can be located by finding the 
highest common factor oif{x) and/'(a;). If the coefficients in the equa¬ 
tions are known exactly (when they will usually be integral), repeated 
roots can be identified with certainty if the H.C.F. process is carried out 
exactly without any rounding off. Otherwise there can generally be 
no certainty whether the equation has a repeated root or two (or more) 
very nearly equal roots. However the H.C.F. process may well establish 
the absence of a repeated root. 

With polynomial equations repeated roots should be removed by 
dividing f{x) by the appropriate product of repeated factors before the 
determination of the remaining roots is started. 

9.55. Division of a polynomial by a quadratic 

A convenient way of finding the complex roots of a real polynomial 
equation with no real roots is to express the polynomial as a product of 
real quadratic factors. In one process for doing this it is necessary to 
carry out a number of divisions by successive approximations to a real 
quadratic factor, and it will be convenient first to consider a numerical 
process for carrying out this division. 

be the dividend polynomial, and 

x^-\-diX-{-d2 

the divisor quadratic. Then we want to find a quotient polynomial 

t For an iterative process for complex roots of polynomial equations, see P. A, Samuel- 
son, Journ, Math, and Phys. 28 (1949), 259. 
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and a remainder such that 

Multiplying out, comparing coefficients, andsolving for qn- 2 .> ^2 

in succession we have 


— 9'o> 

+ 

«2 = 22+<^1</'i + ^45'o- 
03 = g3+<^l?2 + <4<?l. 


S'o ~ ®0’ 

g'l = ttj d^q^, 

g '2 = ^2 d^q^, 

q^ = flg d^q-y, 


2 ?/i—2'f'^l??j—3”f'^2Q^/i,—4> Qn—i 2 3” d^q^—it 

®w-l ~ ^l"l“^l9'n-2 4’^2?n-3> ^1 ~ ®n-l dyq,^_2 d^q^^-^, 

®n ^2~1~^2 2’ ^2 ®n. d^qn—i' 

The numerical work can conveniently be arranged in the following 
scheme: 


Coefficient of 





X 

1 

1 

(Zq 


CL^ 

Otn-2 

^n~i 


-dy 



^i(7i • 

• ^l9n-3 



— 




• 4/ 

(7w- 3;r 

■^2 9n~2 

Sum 




• 9n-2 




On the left the numbers 1, —dy, —d^ are written in three successive 
lines. In the first row are wTitten the coefficients of the dividend poly¬ 
nomial. As each coefficient qj in the quotient polynomial is determined, 
it is multiplied by —dy and —(^ 2 , the product —dyq^ being written in 
the second fine in the column next to that containing q^, and —d^q^ in 
the third fine of the column next but one, as shown by the arrows. The 
sum of the entries in the column containing a,- is the corresponding qj. 

This process, in which the division is started at the highest power of 
X and the remainder is of the form riX-l-r 2 , will be called ‘forward 
division’. A similar process can be carried out starting at the term 
independent of x and proceeding from lower to higher powers of x, with 
a remainder RyX^-^-\-R(yX'^. This will be called ‘backward division’. 
In this process it is convenient to take the divisor as 
and to place 1, —Dy, — D^ on the right and work from right to left, thus: 


Coefficient of x” 




X 

1 


-D^Qi 

ay 

-DyQ, 

-AQ> 

(X^ * • 


1 

-AQ« 

«n 

1 

-Dy 

-A 

Rd 

Ry ~T^Q2 • • 

• Qn-2 



Sum 
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Example: To divide x«+5x*+12x»+14x+8 by x*+2x+4 and by l+fx+^x^ 


Coefficient 

of X* X® X® 

X 

1 

X* X® 

X® X 1 

1 1 5 12 

-2 -2 -6 

14 

/iL 

8 

.—-8 

1 5 

12 14 8 1 

-3-12 -i 

Quotient 
x®-f 3x-f2 

"-2 0 
Remainder 
-2x 

-i -1 

Remainder 

-|x®-§x« 

■^5^—^2^8 Sum 

Quotient 

8 + 2x + 5x2 


9.56. Real quadratic factors of a polynomial 

If some of the roots of a real polynomial equation f{x) — 0 are 
complex, it is best first to determine the real roots Xj, x^ by the 

methods already considered, and remove them by dividing f{x) by 
{x—Xi){x—x^...{x—Xj^y, repeated roots, if any, should be identified as 
explained in § 9.54, and taken out with the corresponding multiplicity. 
This process provides a good check on these roots, since if the remainder 
in the division differs from zero by a greater amount than the tolerance 
for rounding errors, one of the roots must be in error. 

The quotient will be a polynomial F{x) with no real zeros, and its 
complex zeros can be determined by finding its real quadratic factors. 
We take a trial quadratic = x^-yb^x-^c^ and find the quotient Qq 
on forward division of F{x) by Dq (the remainder is irrelevant). Then we 
find the quotient Q% on backward division of F{x) by Qq (or a multiple 
of it), write 

Dj = Qo/(coeff. of x^ in QJ) = x^-\-biX-\-Ci, 

and repeat the process with in place of Dq. This provides an iterative 
processf in which the successive quadratics 

Dq = a;2+6oa;+Co, 

-Oi = a;2+6ja;+Ci, 

converge to the quadratic factor corresponding to the roots of smallest 
modulus of the equation F{x) — 0. The condition for convergence is 
that these roots should be of smaller modulus than any other roots of 
the equation; one of the reasons for removing the real roots is to ensure 
that there shall be no real root of smaller modulus than any of the complex 

t B. Friedman, Commun. on Pure and Appl. Math. 2 (1949), 195. For another itera¬ 
tive process see A. C. Aitken, Proc. Roy. Soc. Edin. 63 (1961), 174. 
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roots, as the presence of such a root would make the results of this 
iterative process oscillate instead of converging. 

Example: To find the real quadratic factors of 

+ 12x2+ 14x + 8. 

For a quartic the method of § 9.52 using equations (9.14) and (9.15) would be 
simpler than the method of this section. The use of a quartic in this example is 
for illustration only. 

The first steps, starting from .x2 + 2x+4 as trial quadratic factor, have already 
been carried out in the example of the last section. The quotient of the first forward 
division is x2 + 3x + 2 = 2(1+ fx + ^.r2); the factor 2 is taken out in order that the 
constant term in this divisor should be unity, and l + fx + Jx^ used as the divisor 
in the backward division. For the present purpose the remainders are irrelevant, 
and the two pairs of columns in the centre can be omitted, so that thus far the 
working could be written 


Coefficient of 



X2 

X2 

X 

1 


1 


1 

5 

12 

12 

14 

8 

1 ) 

-2 



-2 

-6 

-3 

-12 


-1 h 

— 4 




— 4 

— 4 



-i) 



1 

3 

2 

5 

2 

8 




2(J 

i 

1) i 

= 5(1 

0-4 

1-6) 







\ 



The calculation proceeds as follows:—■ 


Coefficient of 


x^ 

X2 

X2 

X 

1 


f 1 

1 

5 

12 

12 

14 

8 

1 ) 

-0-4 


-0-4 - 

-1-84 

-5 

-4 


-0-5h 

.-1-6 



-1-C 

-0‘8 



-0-lJ 


1 

4-6 

8-56 

0-2 

10 

8 


= 8*56(0-l 

0-5 

1) 

= 6-2(1 

1-6 

1-3) 




—— 






Coefficient of 

x^ 

x^ 

X2 

X2 

X 

1 


1 

1 

5 

12 

12 

14 

8 

1 

CD 

1 


-1-6 - 

-5*44 

-5-72 

-5-20 


-0-65 

- 1-3 



- L3 

-1-52 



-0-19 


1 

3-4 

5-26 

4-76 

8-80 

8 


= 5-26(0l9 

0-65 

1) 

= 4-76(1 1-85 

L68) 


Coefficient of 

x^ 

x’ 

X* 

X2 

X 

1 


1 

1 

5 

12 

12 

14 

8 

1 

-L85 


• 1-85 - 

-5-83 

1 

00 

00 

-5-62 


-0-702 

— L68 



-1-68 

-1-78 



-0-223 


1 

3-15 

4-49 

4-34 

8-38 

8 


= 4-49(0-223 

0-702 

1) ; 

= 4-34(1 1-931 1-843) 


The last three approximations to a quadratic factor are 

x2+l-6x+L3, 
x2 + L85x+L68, 
x2+l-931x+l*843. 

Q 


6013 
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The process is first order, so we can use the method of ‘exponential extrapolation* 
(sec §§ 3.4 (a) and 9.32) to estimate a better approximation from these. If we write 
the quadratic x^-\-bx + c, with suffixes 0 , 1 , 2 for these approximations, and use 
formula (9.6), we have 

b^-b^ = 0081, 62-261 + 60 = -0-169, 

extrapolated 6 = 1-931= 1-970; 

C 3 —Cl = 0-163, Cg —2ci + Co = —0-217, 

^ (0-163)2 

extrapolated c = 1-843+ ~ 1*965; 

and the calculation can be continued from these values in a similar way. An 
alternative method of improving the approximation to a real quadratic factor will 
be considered in the following section. 

Notes: (i) As illustrated in the working of this example, only a few significant 
figures need be kept at first when the approximation to a quadratic factor is still 
only rough, and more kept as the calculation proceeds. 

(ii) When applied to a 27ith degree polynomial, the quotient of the forward 
division is a polynomial of the ( 2 n — 2 )th degree and in the backward division we 
have to divide by this quotient. But the quotient of the backward division is a 
quadratic and is determined by the leading three terms in the divisor in this 
backward division, so that the above process for division by a quadratic can still 
be used. 

(iii) There is no accumulation of rounding errors, since at each stage the original 
polynomial is divided by the current trial quadratic factor. 

9.57. Second-order process for improving the approximation 

to a quadratic factor 

The following is an extension of the Newton-Raphson process to the 
improvement of an approximation to a real quadratic factor of a real 
polynomial f{x). 

Let {x^-\-bx-\-c) be an approximate quadratic factor and let 

f{x) = {x'^-\-bx-]rC)q(x)-\-rx-\-s, (9.16) 

where q{x) is the quotient polynomial on di\asion oif{x) by {x^-{-bx-\-c), 
and (rx-t-s) is the remainder. These can be found by the method of 
§ 9.55. Differentiation of (9.16) with respect to 6, for constant x and c, 
gives the variation of the coefficients r and s in the remainder with 
variation of the coefficient 6 in the trial quadratic factor: 

so that —{8rj8b)x—{dsldb) is the remainder when xq{x) is divided by 
{x^-\-bx-\-c). Similarly differentiation with respect to c gives 

0 = 
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SO that — {drj8c)x — (ds/dc) is the remainder when q{z) is divided by 
{x^-\-bx-^c). These remainders can be found by the method of § 9.55 
(the quotients are also found, but are irrelevant), so the partial derivatives 
of r and s with respect to b and c can be determined. 

If now changes A6 and Ac are made in b and c, the first-order changes 
in r and $ are 

and we want to choose A6, Ac so as to reduce r and s to zero, that is, to 



These determine A6, Ac and hence a better approximation 

x^-\- (6+A6)a;-j- (c+Ac) 

to the quadratic factor sought. The process can be repeated, and is 
second-order. 


Example: To improve the approximation l-970x-t-1-965 to a quadratic factor 
of 5x®-|-12x*-|-14x-f-8 (see example in previous section). 


Coefficient of 

x'^ 


X 

1 



1 


1 5 

12 

14 

8 



-1-970 


-1-970 

-5-9691 

— 8-0098 




-1-965 



-1-965 

— 5-9540 

-7-9895 





1 3-030 

4-0659 

0-0362 

0-0105 


Quotient 

q{x) — x^4-3*030x + 4*0659 

r 

s 


Coefficient 

of 

x3 


x"^ X 

1 

x^ 

X 

1 

1 

II 

1 

3-030 4-066 

0 

q(x) = 1 

3-030 

4-066 

-1-970 



1-970 —2-088 



-1-970 


-1-965 



-1-965 

-2-083 



-1-965 


1 


1-060 0-013 

-2-083 ; 

1 

1-060 

2-101 




-S) 

-f-1 

\dbJ 


-d) 

-(D 


Hence A6, Ac are given by 

0-013A64-1-060AC = 0-0362, 
-2-083A6-I-2-101AC = 0-0105, 

and solution of these equations gives A6 = 0-0291, Ac = 0-0337, whence 

b ^ 1-970-I-0 0291 = 1-9991, 
c = 1-965-1-0-0337 = 1-9987, 

so that x*-|- l-9991x-f-1-9987 is a better approximation to a quadratic factor. 
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Actually the quadratic factors in this case are x^-f2x+2 and x*-f 3x-f 4 exactly. 
One application of this method has improved the approximation to the factor 
x*-f2x-}-2 by a factor of about 30. 

9.6. Simultaneous non-linear equations 

For simultaneous equations in two variables the same general pro¬ 
cedure as for equations in one variable can be used, namely a graphical 
process for locating the roots approximately, followed by a numerical 
process for improving the approximation. 

Let the equations be 

/i(^.y) = 0, f^{x,y) = (). (9.17) 

If both of these can be solved formally for y as a function of x, or for x as 
a function of y, then it is easy to draw a graph of y against x for each 
equation, and the intersections of the two graphs give an approximation 
to the solutions. If one or both of the equations can be solved formally 
for y as a function of x or vice versa, then one of the variables can be 
eliminated and the equations reduced to an equation in one variable; 
for example if the second of equations (9.17) can be solved in the form 
y = substitution of this in the first equation gives 

^i(^) = /i(^. = 0. 

There is no need to carry out the elimination explicitly in such a way 
as to exhibit jPi(:r) formally as a function of x\ all that is wanted is that 
y = should be evaluated for a set of values of x, and that these 
should be substituted into the formula for/^(:r, y) for the corresponding 
values of x. This process carries out the ehmination numerically without 
it having to be expressed formally. 

Example: sinx + ^sinT/ = 1, 

2 sin 3x +3 sin 37/ = 0-3. 

It is most convenient here to solve the first equation for sinT/, then from this 
to calculate sin 2y either from the formula 

sin 37/ = sin7/(3 — 4 sin 27 /) 

or by use of inverse sine and sine tables, and then to evaluate 

/ 2 (x) — 2 sin 3x + 3 sin 3y~0*3 

for these values of sin 37/ and the corresponding values of sin 3x. The work is 
conveniently arranged in tabular form. 
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sin 3 / 

sin iy 

sin 3x 

Ux) 


= i(l—sinx) 




0 

0-5 

+ 1 

0 

-f 2-70 

II 

CO 

o 

o 

0-25 

0-688 

1 

3-76 

2(^) = 60° 

0067 

0-200 

0 

-fO-30 

3(i77) = 90° 

0 

0 

— 1 

-2-30 

4(l7T) = 120° 

0-067 

0-200 

0 

-fO-30 

6(^77) = 150° 

0-25 

0-688 

1 

3-76 

6(^77) = 180° 

0-5 

1 

0 

-f2-70 

O 

O 

II 

'h 

0-75 

-f 0-562 

-1 

-0-61 

8(^77) = 240° 

0-933 

-0-450 

0 

-1-65 

9(177) = 270° 

1 

-1 

1 

-1-30 

lO(^) = 300° 

0-933 

— 0-450 

0 

-1-65 

11(J77) = 330° 

0-75 

-f 0-562 

— 1 

-0-61 

12(^) = 360° 

0-5 

1 

0 

+ 2-70 


Two decimals arc adequate to locate the roots approximately. A graph dra'nm from 
these values, or even inspection of the table without actually drawing a graph, 
shows that there are roots in the neighbourhood of xK^n) = 2-1, 3-9, 6-8, and 11-2. 

The approximate solutions so determined can be improved by tabula¬ 
tion at smaller intervals and inverse interpolation, or by an iterative 
process. If both the equations can be solved for one variable in terms 
of the other, say for y in terms of x: 

y = <f>i{x) for the first equation, 

y for the second equation, 

then it may be more convenient to evaluate (f>i{x)—(f> 2 {x) as a function 
of X and interpolate for the zero of this function. 


Example: To find more exactly the root of 

sina:4-2siny = 1, 2sin3a:-l-3sin3y = 0-3 
in the neighbourhood of xK^tt) = 2*1 



x° 

siny 

y = 

sin 3?/ y = 





1 

II 


= 0-1 — § sin 3x 





2-0 

60° 

-0670 

-0671 

-1 

-0334 

--0337 

240 


2-05 

6ir 

-0606 

-0607 

-1523 

-0510 

--0097 

238 

-2 

2-1 

63° 

-0545 

-0545 

-2043 

-0686 

+ 0141 

233 

-5 

2-15 

641° 

-0487 

-0487 

-2556 

-0862 

+ -0374 

231 

-2 

2-2 

66° 

-0432 

-0432 

-3060 

-1037 

+ -0605 



and inverse 

interpolation 

then gives 

the required solution, approximately 


X = 2-070(^77). 
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When neither of the equations can be solved formally for x or y, the 
same processes can be used, one or both of the functions <f>i{x), <f> 2 {x) 
being determined roughly graphically, or more accurately numerically, 
by solution of the equation /^{x, y) — 0 orf 2 {x, y) = 0 for y in terms of x. 
For example, if a set of graphs oif^{x, y) against y for a set of constant 
values of x is constructed, the intersections of these graphs with the 
y-axis give the function y = <j)i{x), which can then be used to substitute 
for y in the second equation. 

Another process is to evaluate fy{x, y) and y) for a set of points 
on a coarse grid in the {x, y) plane, and on a piece of squared paper to 
mark at each {x, y) point the values of fi{x, y) and / 2 (ir, y) there. The 
loci/j(a;, y) = 0 and f 2 {^,y) = 0 can then be sketched roughly, and the 
intersections of the curves thus sketched then indicate the regions of 
the plane in which a closer examination is necessary in order to determine 
the roots more accurately. 

Example: To locate approximately the real solutions of 

xy{2x^ — y^)-\-\Q(x-\-y) = 48, (9.18) 

a;2 4-2/2 = 16. (9.19) 

The second equation shows that x and y lie between 4 ; 4, so evaluate 

xy{2x'^—y^) + lQ{x-\-y) 

on a square grid of mesh side unity in the (x,y) plane for |a;| < 4 , |y| < 4 (see 
Fig. 16). Although this grid is a coarse one, it enables the contour fi{x,y) = 0 , 

xy{2x^-y^)+16{x+y} = 48, 

to be sketched roughly. In this particular case the contour fi(x,y) = 0, that is, 
a; 2 -f 2/2 = 10 ^ could be drawn accurately; but in the figure it has been sketched 
freehand from the values of mesh points, as would have to be done 

in general. 

The intersections of the two contours show that there are four real solutions, 
approximately; 

X = —2-0 X = 0-4 a; = 1-8 x = 4-0 

y = 3-5 y = 4-0 y = 3-6 y = -0-2 

and probably two in the neighbourhood of x = —3 5, y = —0 9, though calcula¬ 
tion of function values on a finer grid would be necessary in order to make certain 
whether the contours intersect in this region. 

Notes: (i) The values recorded in the figure are not those of/i(a;,y) and/ 2 (x,y) 
themselves, but are those of the left-hand sides of equations (9.18), (9.19). 

(ii) Since/i(x, y) in this case is a cubic in x for fixed y, and a cubic in y for fixed 
X, the table of its values can be checked both very easily and very thoroughly by 
differencing in both directions. Alternatively, these values could be built up from 
the differences in the y direction and checked by differencing in the x direction 
(or vice versa). 

(iii) Since this method involves evaluating the function on a twofold array of 
points, it should in general be avoided when it is possible formally to use either 
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Fig. 16. 


of the equations to solve for one or other variable in terms of the other, and so to 
reduee the problem to one in a single variable. It is used in this example to pro¬ 
vide the possibility of comparing this method, and the results of using it and of 
improving the solutions, with others. 

A eonvenient alternative method for these equations is to use the substitution 
X = 4eos0, y = 4sin0, whieh ensures that the equation (9.19) is satisfied, and 
then to treat equation (9.18) as an equation in 6. Another alternative is to use 
equation (9.19) to substitute for y’^ in (9.18), whieh then becomes 

[{3x^-l6)x+l6]y = 4S-16X, (9.20) 

and then treat equations (9.19) and (9.20) as two equations of the form y = ^(x), 
(iv) The best method of improving the approximate solutions will depend on 
the equation and may be different for the different solutions. In the present 
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example an iterative process is most convenient for the solutions near x — 0-4, 
y = 4'0,anda; = 4'0,j/ = —0-2. Consider the latter. We use equations (9.19) and 
(9.20) alternately, the first to determine x from an approximate value of y, and the 
seeond to determine y from an approximate value of x. Since \ylx | is small for this 
solution, the value of x obtained from (9.19) is insensitive to the value of y taken, 
and is not much altered when an improved approximation to y, derived from 
(9.20) with this value of x, is used. The iterative process, though only first order, 
converges rapidly. 

A similar treatment, using an equation obtained by substituting for a;* instead 
of for j/* in (9.18), is similarly effective for the solution near x = 0-4, y — 4*0. 

The following is a general process for improving the approximate 
values of a solution, when neither equation can he solved formally for 
either variable in terms of the other. The functions fi{x,y), f2,{x,y) 
are evaluated on a finer grid of points (a:, y) in the neighbourhood of the 
solution. But instead of the function values being recorded in the {x, y) 
plane, the value of f2{x, y) is plotted against f^ix, y) for each pair of 
values {x,y), and curves of constant x (x-contours) and of constant y 
(y-contours) are drawn in the (/i,/ 2 ) plane. The advantage of this 
method of representing the behaviour of the two functions of x and y is 
that each curve is drawn through plotted points instead of being inter¬ 
polated ‘by eye’ among an array of function values. 

Such a plot of/g againstis made for such values of x and y that the 
point = f^ = 0 is enclosed between two a:-contours and two y-contours. 
Within a small enough region not containing more than one solution, 
the a:-contour8 and y-contours will usually be nearly equally spaced and 
not very curved, and if this is the case, it is possible to estimate fairly 
closely what contours pass through the point = f^ = 0. A calculation 
of ifiJi) for this approximation to the solution then suggests for what 
further values of {x, y) the function should be evaluated in order to enclose 
the point = /g = 0 still more closely. The process is illustrated by 
the following example. 

Example: To find more accurately the solution of the equations 
a;t/(2a;»—y*)-f-16(a;-l-y) = 48, = 16, 

in the neighbourhood of a; = 1-8, y = 3-6. 

From Fig. 16 it is estimated that the solution lies between a; = 1-7 and 1-9, and 
between y = 3‘5 and 3-7. The improvement of this solution can be carried out 
by the following procedure. A set of values ofand /j is first evaluated for y — 3-5, 
3-6, and 3-7, x = 1-6 to 2-0, this range of a; being taken in order to provide enough 
values to check by differences. The a;-contours and y-contours drawn using these 
points are shown in Fig. 17; those for y = 3-5 and 3-6 already enclose the point 
/i — ft = 6; but some points for y = 3-7 have been calculated to check the spacing 
of the y-contours and to show the curvature of the x-contours. The (fi,fi) point 
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for X = 1*85, y = 3*55 is also shown. The solution estimated from these contours 
was X — 1-84, y = 3*55. 



Values of (/ij/g) are now calculated for these four sets of values x = 1*83, L84 
and y = 3-55, 3*56, and the results plotted on a larger scale. For this small range 
of a: and y^ the contours can be taken as straight and equally spaced to the accuracy 
of the plot. Inverse interpolation in x and y is required to determine the values of 
X and y to give — f^ — 0, and this is most easily done by measurement. The 
values obtained can be checked by calculatingand /2 for them. 

Notes: (i) As in Fig. 16, the functions plotted in Fig. 17 are the left-hand sides of 
the two equations, namely/i(a:, 2 /) + 48 and/ 2 (ir, 2/)+16. 

(ii) A convenient way of carrying out the final interpolation is[[as follows. Con¬ 
sider the interpolation between the two ic-contours, say x = x^ and x^y and let 
X = Xq-\-p{x^—Xq) be the interpolated value required. Lay a ruler on the (/ij/z) 
diagram so that its edge passes through the point/^ = /g = 0; rotate it about this 
point and move it in the direction of its length until it cuts both ir-contours at 
exact graduations on the scale, at a convenient interval (say 5, 10, or 20 units of 
the scale graduation); then the value of the fraction p of the x-interval between 
the contours can be read off directly. 

(iii) This method of plotting is not satisfactory for the preliminary location of 

roots, since (x, y) is not in general a single-valued function of so that two 

or more x-contours and two or more y-contours may pass through each point in 
a region of the (fiyfz) plane. If this occurs, and it will occur if the equations have 
more than one solution, the diagram becomes complicated and its inter¬ 

pretation needs considerable care. 

9.7. Three or more variables 

There is no satisfactory practical method, graphical or tabular, of displaying the 
behaviour of functions of three or more variables, and the approximate location 
of solutions of such equations is therefore difficult. 
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In some cases the solutions can be regarded as representing the asymptotic 
steady-state values of the solutions of a system of differential equations, and then 
they' may be determined approximately by integrating this system of differential 
equations. If the equations arise from some scientific problem, this may suggest 
the appropriate differential equations to use. For example, in the chemical equili¬ 
brium of a system of a number of gaseous components, the relations between the 
concentrations of the components are given by a set of non-linear equations in¬ 
volving the equilibrium constants of the various reactions. If, for example, one 
of the reactions was 2 CO + O 2 ^ 2 CO 2 and ‘^(X) stands for the concentration of 
the molecular species X, one equation would be 

[^(C0)]2^(02) = (9.21) 

But the equilibrium is attained through a non-steady process in which the con¬ 
centrations of the components change with time, that of oxygen, for example, being 
given by an equation 

I nO^) = «’(0,){«’(C0)}*]. (9.22) 

and similarly for other components. We can try to make the calculations approach 
a steady state by following out such a time-varying process. However, since 
the purpose of the differential equation (9.22) is solely to provide a means for 
approaching a solution of the equation (9.21), there is no need to take experimental 
values of the reaction rate coefficients like in (9.22) even if these are known; 
an artificial set can be taken, convenient for the numerical work, and they need 
not even be taken to be constant. 
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FUNCTIONS OF TWO OR MORE VARIABLES 

10 .1. Functions of a complex variable and functions of two 
variables 

There are two rather distinct contexts in which functions of two or 
more variables may arise in numerical work. One is concerned with 
complex numbers and functions of a complex variable. In numerical 
work it is usually best to treat a complex number as a pair of real numbers, 
either {x, y) in the Cartesian form (2 = x-\-iy) or {r, 6 ) in the polar form 
(2 = of the complex number as is most convenient for the calcula¬ 
tion concerned. In this context a particularly important feature is the 
property of any analytic function f{z) — g{z)-\-ih{z) of a complex 
variable 2, that its real and imaginary parts both satisfy Laplace’s 
equation in two dimensions. For this reason, the finite difference form 
of the two-dimensional Laplacian operator {d^ldx^)-\-{d^ldy^) plays a 
particularly important part in such contexts. 

The other is the general case of functions of two or more real variables 
other than those arising from formal expressions involving complex 
numbers. Here, too, the finite difference form of the Laplacian operator 
is important, particularly in two dimensions, and in three dimensions 
with some degree of spatial symmetry. 

10.11. Numerical calculations with complex numbers 

The details of numerical calculations with complex numbers will be 
carried out almost entirely with pairs of real numbers, since there is no 
standard calculating machine which deals directly with complex 
numbers. For addition and subtraction the Cartesian form 2 = x-\-iy 
is clearly the more convenient. For multiplication and division the polar 
form 2 = |2 seems preferable to the Cartesian form since although the 
Cartesian formulae 

{x^-{-iyi){Xi-\-iyi) = x^x^—y^y.^+i{x^y^-\-x<^y^), 

{x^-{-iyi)l{Xi-[-iy2) = [xiX^-\-yiyi-\-i{—Xiyi-\-x^y^)]l{xl-\-^jl) 
are not' difficult to evaluate, it is also not difficult to make a mistake of 
sign in this evaluation, particularly when x^, x^, y^, and y^ are not all 
positive. Use of the polar form will probably involve some conversion 
from Cartesian to polar form; various good modern books of tablesf 
I For example, Chambers''a 6-Figure Tables, voL 2 (1949). 
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include tables for simplifying this conversion. Whether, and at what 
stages of a calculation, it is advisable to make a conversion from Car¬ 
tesian to polar form or vice versa will depend so much on the calculation, 
and also perhaps on the individual worker and on whether Cartesian- 
polar conversion tables are available, that no general rule can be laid 
down. 

For finding powers (other than squares and perhaps fourth powers) 
or roots of complex numbers, the polar form is usually the most con¬ 
venient. But square roots can be found directly from the Cartesian 
form as follows. Let ^ 


where r] are real. On squaring and separating real and imaginary 
parts this gives = x, 2 ^rj = y. (10.1) 


Elimination of rj gives a quadratic for of which only the positive 
root is significant since ^ is real; this root is 

whence 


I = |(x2-|-y’-)i|}]i, y = [i{-x-M(.'r2+y2)i|}]l, (10.2) 

the signs of these square roots being taken so that 2 ^ 7 ] = y. If x is 
positive it may be best to use the first of formulae (10.2) to determine 
and then to find y from y = y/2^; and sirnilarly if x is negative to use 
the second of formulae (10.2) to determine y, and then find ^ from 
^ = yj2y. The result can be checked by squaring the value of (^+*17) 
obtained. 


10.2. Finite differences in two dimensions; square grid 

Just as for functions of one variable x we often have to consider func¬ 
tions as specified by a table at discrete values of x, usually at equal 
intervals, so for a function of two or more independent variables we are 
often concerned with a function specified at discrete, equally spaced, 
values of all the independent variables. In particular, with two inde¬ 
pendent variables {x, y) it is very often most convenient to take these 
discrete values of x and y in such a way that they form a grid of square 
mesh in the {x, y) plane, such as 

(x, y) = {Xf,-{-j Sx, yo+A: Sy); Sx = Sy = 8 s (10.3) 

with integral values of {j,k). The values of a function / at such a point 
will be written 

Such a function can be differenced in the x direction and in the y 
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direction; 8 y will be used for central difference operators in the x and 
y directions, so that 

^xf},k f}+i,k fj—\,k'> ^vfj,k fj,k+i fj,k-i> 

^xfj,k — fj+l,k~^fjk'^fj-l,k’ ^vfj.k —fj,k+l 2/yfc+/;,fc-i- 

A particularly important relation is 

{^l-^^l)fj,k =/;+l,fc+/;.fc+l+/j--l,fc+/;,fc-l —( 10 . 4 ) 
The operators ^x{dldx) and 8 y{djdy) w^ill be WTitten C 4 , Uy, the notation 
being an obvious extension of that of § 4 . 7 . Then, as in § 4 . 7 , 

83 . = 2 sinh I U^, By = 2 sinh | Uy, 
and S|-i-Sy = 2(cosh C4+cosh C/j,) —4. (10.5) 

It is convenient to represent formulae such as ( 10 . 4 ), which represent 
linear combinations of values of / at a set of 
neighbouring points in the (x,y) plane, in a 
diagrammatic form in which the way in which 
the different function values enter is more 
immediately evident. Bickleyf uses diagrams 
in which the set of coefficients in formula 
( 10 . 4 ) would be represented by Fig. 18 . A 
similar diagram, how ever, is used by Southwell 
in a different sense (see § 10 . 61 , Fig. 19 ), and 
its use to represent the coefficients in formula 
( 10 . 4 ) might be confusing. A more convenient form for printing is the 
following diagrammatic representation of formula ( 10 . 4 ): 



(S| + S^)/y.. = 


0 

1 

0 

1 

— 4 

1 

0 

1 

0 




the set of coefficients being enclosed in a ‘box’ to distinguish it from a 
matrix. Another formula which will be needed, and which can be 
written in a similar form, is 





1 

— 2 1 

8 |S5/„ = 

1 -2 1 

= 

_2 

4 -2 




1 

— 2 1 

quantity (Sl+S^)/^^ is four times the 

difference 1 


Ak- ( 10 . 6 ) 


arithmetic mean of the values off at the corners of a square centred on 
the point (j, k) and the value off at the centre, the corners of the square 


I W. G. Bickley, Quart, J, Mech, and Applied Math, 1 (1948), 35. 
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being the grid points which are the nearest neighbours of {j, k) (the side 
of the square is V2(8s), not 8s). A similar quantity involving next- 
nearest neighbours is 


1 

0 

1 

0 

—4 

0 

1 

0 

1 


fj,k — "^hk- 


In terms of the operators C 4 , Uy this is 

[e u,+c/,q_ e- c/,+u, e- u,- 4 ] 

= [2 cosh(C7,+ C7J+2 cosh(C7,- C/„)-4]/,., 
=: 4[cosh C4cosh Uy—l]fj-,,. 

In terms of the differences of/it can be written: 


1 

0 

1 


( 

1 

_ 9 

1 


0 

2 

0 

0 

—4 

0 

fi.k — ' 


— 2 

4 

— 2 

+ 

2 

— 8 

2 

1 

0 

1 


\ 

1 

—2 

1 


0 

2 

0 




so that 8l8lfj. ,,= 


[8182 + 2 ( 81 + 82 )]/,,,. 

/,,,-2(82+S^)/;-... 


1 

0 

1 

0 

— 4 

0 

1 

0 

1 


(10.7) 


10.3. The operator d^/dx^-\-d^/d-y^ 

The particular importance of the Laplacian operator in two dimensions 
has already been noted in § 10.1. On a grid of square mesh of side 8s 

we have {8s)%d^ldx^-\-d^/dy^) = C/ 2 +C 72 , 

and are therefore concerned with finite-difference approximations to 
Using the approximations of §4.71 we can express this in 
terms of the operators 8^ and 82 as follows: 

Ul+t'S = S?[(sinh-‘i8j)/J8J=+S*[(sinh-‘ 

= 8i-ftSJ+0(8j:)=+85-^S}+0(Ss,)= 

= S|+85-i(8|+S5)+0(8«)«. (10.8) 

Thus the simplest approximation to C/ 2 + 1/2 is 

Ul^rUl = 


which gives n + m 

\0x2 ay2/,,,. 




0 

1 

0 

1 

— 4 

1 

0 

1 

0 


k 


i.k 


(10.9) 
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with an error term of order ( 85 )^. This approximation is widely used 
in numerical work. In particular it gives 


0 1 0 


1 -4 1 = 0 


( 10 . 10 ) 


0 1 0 


as a finite-difference form of d^fjdx^-\-d^fldy^ = 0 . 

10.31. Special relations when d^fldx^-\-d^fldy^ = 0 
In many contexts in which the operator {d^ldx^-\-d^jdy^) arises, its 
importance comes from the fact that one or more of the functions f{x, y) 
concerned satisfy the relation 


djjdx^^d^fjdy^ = 0 . 


This is always the case when we are concerned with analytic functions 
of a complex variable, and is often the case in calculations not directly 
concerned with complex variables. If the operands are restricted to 
such functions, we have 



and this can be used to obtain some special formulae for use in such con¬ 
texts; but it must be remembered that they are restricted to such 
operands. 


One of the most important can be derived as follows: 
Since = 0 it follows that 


Hence 


U%=Ul=-UlUl. 

- - 28|82 + 0 ( 85 )«, 


( 10 . 11 ) 


so that formula ( 10 . 8 ) can be written 
and substitution from (10.7) gives 


1 0 1 


4(8|-f8*)/,.,+ 0 -4 0 0(Ssr 


1 0 1 


that is 


1 4 1 

4 -20 4 = 0 

1 4 1 


( 10 . 12 ) 


with an error term of order ( 8 s)®. This is an improvement on the simplest 
finite-difference form ( 10 . 10 ) of the equation d^fjdx^-^-d^fjdy^ = 0 , for 
which the error is of order ( 8 s)'*. 
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Another consequence of the relations (10.8) and (10.11) is that 

and this can, if convenient, be used in integration or interpolation 
formulae to substitute for fourth differences in the x direction in terms 
of the second differences. For example, one formula for integration 
in the x direction is 

xo + 8x 

J fdx = 2(8x)[fQ-}-^S^fo—(,]-}-0(Sxy (10.14) 

(this is equivalent to Simpson’s rule improved by the inclusion of the 
leading correcting term; see § 6.3), Expressed in diagrammatic form 
in terms of function values, this is 
xj+i 

f f(x, 9 j;,)dx = ^(Sx)[-l 34 114 34 -l]/^,;,+ 0(Sx)’. (10.15) 

Xj^l 

Substitution for S*/ from (10.13) in (10.14) gives 

f fdx = 2(Sx)[/„+iS}/„-iSi/„]+0(8xy. 

xo — 8x 
1 

that is,t J f(x, Vk) dx = ^(Sa:) 

The coefficients are simpler in (10.16) than in (10.15) and the coefficient 
in the error term is smaller, as might be expected from the fact that the 
values of / involved in formula (10.16) lie nearer the range tlirough 
which the integration is being carried than do the function values in 
(10.15).$ 

10.4. Finite differences in cylindrical coordinates 

It is occasionally convenient to use finite differences at equal intervals 
in polar coordinates (r, 6) in a plane, or in cylindrical polar coordinates, 
rather than in Cartesian coordinates. Plane polar coordinates would 
be the natural ones to use, for example, in a calculation concerned with 
a solution of Laplace’s equation in two dimensions with boundary 
conditions given on a circular boundary; and cylindrical polar co- 
orcUnates would be the natural ones to use in a three-dimensional problem 

•f This formula was first derived by another method by G. Birkhoff and D. M, Young, 
see Journ, of Math, and Phys. 29 (1950), 217. 

X For a similar use of the relation — Oin the interpolation of functions of a 

complex variable, see P. M. and A. M. Woodward, Phil. Mag. (7) 37 (1946), 236; 39 
(1948), 594. 


0 

— 1 

0 

4 

24 

4 

0 

— 1 

0 


/,.,^+0(8a;)7. (10.16) 
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wjth axial symmetry and boundary conditions on the surface of a 
circular cylinder. These cases can be considered together, the case of 
plane polar coordinates being given by putting d/dz = 0 in the equations 
for cylindrical polar coordinates. 

One way of dealing with such calculations is to make the conformal 
transformation to (log r, 6) and to work on a rectangular or square grid 
in the (log r, 6) plane. But this is often not convenient when the point 
(or axis) r = 0 is in the domain to be covered by the integration, and it 
is then better to use the (r, 6) coordinates without modification. 

Consider first the case of axial symmetry. Then the Laplacian operator 
in cylindrical polar coordinates is 

32 Id 32 

3j.2 ^ 3z2 


The finite-difference approximation to d^jdz^ is the same as in Cartesian 
coordinates; only the r-derivatives need special treatment. Let/^ stand 
for f{j Br). Then 


(0) = +/,-.)/(8--)*+ 0(8r)*. 

An approximation to {dfldr)p with an error term of the same order, is 


SO that for j ^ 0 


= [(2j-l)/,--i-4i/,.+ (2i+l)/,J/2i(8r)2+0(8r)2. (10.17) 


For axial symmetry, either there is a singularity at r = 0 or 3//3r is 
zero there. If there is a singularity, further analytical investigation is 
required before numerical methods are applied. If 3//3r = 0 at = 0, 

then (|Y+i|A =4(/,-/„)/(8r)>+0(Sr)^ (10.18) ’ 

\3r- r 3r/o 

a relation which can also be obtained from (10.9), since for axial sym¬ 
metry each of the values of / with coefficient unity in (10.9) is/j. 

If there is not axial symmetry, then there is an additional term 
r- 232/302 in the Laplacian operator, and if/,- ^. stands for/(j Br, k BO), we 
have for j ^ 0, 


/I 32n 

1 - M 


\r2 302 J 

[■ic O' Sr)"l 

— 1 

_1 


+ 0{Bef. 


(10.19) 


6018 


R 
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For the equation for^* = 0, let/j be the arithmetic mean of the values 
fi,k of / oo the circle r = Sr. Then 

(0+;l+r^0).= 00-20) 

If / varies in the z direction, then to give V^/, a finite-difference 
approximation to d^fjdz^ has to be added to whichever of formulae 
( 10 . 17 ) to (10.20) is the appropriate one to use for the variations in the 
(r, d) plane. 


10.5. Partial differential equations 

Solutions of partial differential equations can sometimes be obtained 
by a separation of variables, by which the partial differential equation 
is reduced to a number of separate ordinary equations, one in each of the 
independent variables. Such a separation, if possible, is part of the 
preliminary analytical treatment of the problem before numerical 
methods come to be applied, and will not be considered here. The follow¬ 
ing sections are concerned with the numerical treatment of partial 
differential equations as such. It will mainly be concerned with partial 
differential equations in two independent variables, as the numerical 
solution of equations with three or more independent variables is 
usually a problem on too large a scale to handle without special equip¬ 
ment. 

Many partial differential equations which arise in contexts in which 
numerical solutions are required are second order in at least one of the 
independent variables, and, moreover, are linear in the second-order 
derivatives. Simple examples are Poisson’s equation in two dimensions 

3 + 0 =»**’*'>’ 

where g{x,y) is given; the equation of heat conduction or diffusion in 
one dimension 


in which the diffusivity D may depend on/ (this would make the equation 
as a whole non-linear, but the second derivative enters linearly); and 
the wave equation 

( 10 . 23 ) 


8t^ 


Oi 




8x^ 


Just as the nature of the problem of numerical solution of ordinary 
differential equations depends on whether the conditions the solution 
has to satisfy are of the one-point or two-point type, so the nature of the 
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problem of the numerical solution of partial differential equations 
depends on whether the boundary conditions are given on a boundary 
completely enclosing the domain of the variables over which a solution 
is required, or whether this domain is unbounded in one or more direc¬ 
tions. There is a classification of second-order equations in two variables 
as ‘elliptic’, ‘parabolic’, or ‘hyperbolic’ which is closely related to the 
different characters of boundary conditions usually associated with 
such equations, and the character of the problem of numerical integra¬ 
tion is correspondingly different in the three cases. 

The general second-order equation in two variables, linear in the 
second derivatives, is 

where H, K, L, M may be functions of any one or more of the variables 
X, y, f, dfjdx, dfjdy. The classification depends on the sign of K'^—HL] 
the reason for this will be explained in § 10.8. If this quantity is negative, 
the equation is termed ‘elhptic’; if it is zero, the equation is termed 
‘parabolic’; and if it is positive, the equation is termed ‘hyperbolic’. 
Poisson’s equation (10.21) is a simple example of an ‘elliptic’ equation, 
the diffusion equation (10.22) is one of a ‘parabolic’ equation, and the 
wave equation (10.23) is one of a ‘hyperbolic’ equation. ‘Elliptic’ 
equations are usually associated with a domain completely bounded by 
closed curves (one of which may be the circle at infinity) on which 
boundary conditions are given. ‘ParaboUc’ and ‘hyperbolic’ equations 
are usually associated with a domain which is open in the direction of 
one variable, which physically is often the time variable. For example 
we may require a solution of the heat conduction equation (10.22) from 
given initial conditions in time (/ given as a function of a: at ^ = 0) and 
with given terminal conditions in space (/ given as a function of t at 
X = a, X = b) but with no condition to be satisfied at a later time t = T; 
the initial and terminal conditions are enough to define a solution, and 
such an independent condition at a later time could not generally be 
satisfied. It is not, however, necessary that the boundary conditions 
should be of this type; we might alternatively have no initial conditions, 
but given terminal conditions and a condition of periodicity in time, that 
is, a condition that/should be the same function of a: at a given time T 
as at time t = 0. 

If H, K, and L are not all constants, then the equation may be of 
different type in different parts of the domain in which the solution is 
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required. But for many of the simpler partial differential equations, 
such as (10.21), (10.22), and (10.23), including many practically impor¬ 
tant ones for which numerical work is likely to be needed, the equation 
remains of the same type throughout the whole domain, and only such 
cases will be considered here. 

10.6. Elliptic equations 

Poisson’s equation (10.21) in two dimensions will be taken as a typical 
example of an elhptic equation for whose solution we require a numerical 
process. This process will cover as special cases Laplace’s equation 
{g{x,y) = 0 in (10.21)) and the torsion equation {g{x,y) = const.). The 
first step is to replace the partial differential equation by a finite- 
difference relation on a convenient grid of discrete points. A Cartesian 
or polar grid will usually be most convenient, and for the present only 
a Cartesian grid of square mesh with mesh side h will be considered. It 
is clearly most convenient if the boundaries are of such a form that a 
grid can be chosen so that the boundaries he along the sides or diagonals 
of the grid squares, and it will be supposed for the present that this is 
the case and that the grid is so chosen. 

Using the simplest approximation (10.9) to [{d^ldx^)-\-{d^ldy^)'] we 
then have a set of equations 


fi.k = h^9i,k, (10.25) 


one for each mesh point. These are hnear simultaneous algebraic 
equations, so that we have formally reduced the numerical problem to 
one of the kind already considered in Chapter VIII. The solution of 
the set of equations (10.25) is not, of course, the solution of the partial 
differential equation on account of the truncation error of the approxima¬ 
tion (10.9). The approximation can be improved by taking a finer mesh 
or by using the better approximation (10.8) to d^fldx^-\-d^fldy^. If the 
latter process is used, a convenient procedure is to write the finite- 
difference equation 


fi,lc — (10.26) 


0 

1 

0 

1 

—4 

1 

0 

1 

0 


0 

1 

0 

1 

—4 

1 

0 

1 

0 


and to solve this by an iterative process, using in the nth stage of the 
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iterative process values of (8^+8^)/ obtained from the results of the 
{n —l)th stage.t 


10.61. Relaxation process 

A ‘relaxation’ process (§ 8.5) is very convenient for carrying out the 
numerical solution of the set of equations (10.25) or (10.26), and is 
commonly used for this purpose. This common association of the 
relaxation procedure with the approximate equations (10.25) seems to 
have given the impression that the relaxation process itself is approxi¬ 
mate, and the errors of the approximation (10.25) are sometimes referred 
to as ‘errors of the relaxation process’. But this is a misunderstanding; 
the approximation is not in the relaxation process itself but in the 
equations (10.25) w'hose solution is evaluated by this process. Regarded 
as a solution of the partial differential equation, the solution of equations 
(10.25) is equally in error whether it is evaluated by the relaxation pro¬ 
cess or by any other (such as elimination or inversion of the matrix of the 
coefficients of equations (10.25)) and the errors have nothing to do with 
the relaxation process used to obtain a solution of these finite-difference 
equations. 

The approximation to the solution of the partial differential equations 
can be improved by reducing the mesh size of the grid on which the finite 
differences are taken. In practice it is advisable to start with a very 
coarse grid so that the number of grid points is quite small, and then 
to break down the grid to one of smaller mesh size when an approximate 
solution on the coarse grid has been reached. Then the relaxation process 
on the finer grid starts from a set of values which is already a fair approxi¬ 
mation to the solution. 

It is convenient to carry out the relaxation process on a diagram 
representing the domain in w'hich the solution is required, with the finite- 
difference grid drawn on it. The usual convention is to write the function 
values and their changes to the left of each grid point, and values of the 
residuals to the right. For the simplest finite-difference approximation 
(10.25) to Poisson’s equation, the residual at the point (j,^*) is 



0 

1 

0 

1 

—4 

1 

0 

1 

0 




(10.27) 


If a relaxation A/ is made at one point, the residual at that point is 
changed by — 4A/, and that at each nearest neighbour, other than a 


I See, for example, L, Fox, Proc, Boy, Soc, A, 190 (1947), 31. 
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boundary point, is changed by +A/, so that the pattern of the changes 
in the residuals is as represented diagrammatically in Fig. 19. The 
entries here are the coefficients of a single A/ value. 


Example: To find approximately the solution of Laplaee’s equation 

dw 8W _ 

for the system shown in Fig. 20, with equipotentials F = 0 and F = 80 as 
indicated. 







The first, coarse, grid can be taken as shown in Fig. 21. It might seem at first 
sight that this grid is too coarse for the results to be of any value. But we shall 
see that this rough approximation is in fact useful, and is obtained much more 
easily and quickly than results on a finer grid. By symmetry, only half of the 
diagram need be shown, but it must be remembered that each relaxation AF at 
a point one interval from the centre line is accompanied by an equal one at the 
image point, so that the contribution to the residual on the centre line is 2AF. 

A set of values of V from which to start the relaxation process can be written 
ill as if the equipotential F — 0 were the straight line AB, Then the residuals are 
zero except on AB, The.sc values of F and the residuals are entered on Fig. 21. 

We could start the relaxation process by making such a relaxation as to reduce 
the residual at C (for example) to zero; this would require a relaxation AF = + 10 
at C (and at its image in the centre line). But clearly a positive relaxation AF is 
going to be required at D, which will make a positive contribution to the residual 
at (7, and a further positive relaxation AF at (7 will be needed to remove it. We 
can anticipate this by deliberately taking a larger relaxation AF at (7 than is 
required to reduce the residual there to zero; this is called ‘over-relaxing’. Expe¬ 
rience is the only way of learning when and by how much to over-relax; the 
beginner will probably be inclined not to over-relax enough. As a rough rule it 
may be suggested that w^hen there are several residuals of the same sign together, 
over-relaxation by a factor 2 will not be excessive. 
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Let us start, therefore, with a relaxation AF = +20 at (7; this leaves a con¬ 
siderable positive residual at Z>, flanked now by large negative residuals at C and 
its image point, so that a smaller degree of over-relaxation is now required. Let us 
therefore take next a relaxation AF = +25 at D, 

The further process of the calculation is indicated in Fig. 21, the relaxations and 
the values of the residuals resulting from them being indicated by (a), (6), (c),... 
in succession. At some convenient stage in the work, it is advisable to collect 
together the values of AF and write down a new set of F’s at the mesh points, and 
to recalculate the residuals from these values of F in case any mistake has been 
made in the relaxation process. The calculation can then be continued from these 
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values of V and the corresponding residuals. This check should normally be made 
before the accuracy of the numerical work is increased by taking an extra signifi¬ 
cant figure (compare the examples in §§ 8.5 and 8.51) and always before changing 
from a coarser to a finer grid. 

Notes: (i) The individual numerical steps of the relaxation process are very simple 
and are carried out with small numbers, usually of one or two significant figures; 
they can therefore be carried out rapidly and easily. 




(ii) When the value of F at a point not at a distance 8^ from a boundary is 
changed, the sum of the residuals remains unchanged; all that is changed is the 
distribution of this total among the grid points. But if a relaxation AF is made 
at a grid point P adjacent to a boundary, the sum of the residuals is reduced by 
AF, or by 2AF or 3AF if two or three of the nearest neighbours of P are on the 
boundar^^ (see Fig. 22). 

(iii) A physical analogue of the relaxation process, as applied to the finite- 
difference form of Laplace’s equation, can be given by considering Poisson’s 
equation for the potential of a two-dimensional distribution of electrical charge, 
namely 

dy^ 

The finite-difference approximation (10.9) to the left-hand side gives 



(10.28) 


and is (to this approximation) the charge on a square of side h centred on 

the point (j, k). Thus, for any assigned set of values of the residuals are 
a measure of the charge distribution required to give the assigned potential 
distribution. The relaxation process can be regarded as a process of shifting this 
charge distribution about imtil it is ultimately all in the form of surface charge 
on conductors forming the given equipotential boundaries and none is left as 
space charge in the domain over which the integration is carried. 

The constancy of the sum of the residuals when a relaxation is made at a grid 
point not adjacent to a boundary corresponds to the constancy of the total space 
charge in the domain when some charge is taken from one gi'id point and distributed 
among its four nearest neighbours. The change in the sum of the residuals when 
a relaxation is made at a point adjacent to a boundary corresponds to tho transfer 
of some of the speice charge to surface charge on the boundary. 
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This analogy suggests that the aim of the relaxation process should be not only 
to make the residuals small but to make them not all of the same sign, so that their 
sum, represented by the total residual space charge in this analogy, is small. It will 
not in general be possible to reduce all residuals to zero in the least significant digital 
position; a sprinkling of values d: 1 with oeeasional values ±2 is the best that ean 
be expected, and such a set of residuals, with mean value perhaps 0*1 or 0*2, 
probably indicates a better approximation to a solution than a set of residuals +1 
over the whole field. 

(iv) A set of residuals of magnitude not greater than 2 does not necessarily mean 
that the values of V are correct to a unit. It is advisable to reduce the residuals 
on the final grid to i 2 in the next figure beyond the last figure in V required in 
the final results. 

(v) No indications such as the letters (a), (6), (c),... in Fig. 21 are required in 
actual working; they are only given in this figure to help the reader to follow the 
details of the calculation. As soon as one value of a residual is replaced by another, 
the earlier one ean be crossed out or erased as being of no further interest. 

{vi) If the grid is drawn in ink and the working is done lightly in pencil, then old 
values of V and old residuals can be erased without losing the pattern of the grid. 
This erasing of old values need not be done at every relaxation, but only when the 
space for values of residuals gets filled up. 

(vii) In this example the over-relaxation by a factor of 2 in the first relaxation 
has been a little too much, and a small relaxation of the opposite sign has had to 
be made later. But this step of over-relaxation has speeded the approach to a 
solution of the finite-difference equation. Only fourteen steps of relaxation have 
been needed to reduce the greatest value of \ from 40 to 2. 

(viii) With the very coarse grid used here, there is no point in tr^dng to improve 
the approximation to the solution of the finite-differenee equations by taking an 
extra figure in the F-values. The next step is to reduce the truncation errors by 
taking a finer grid. 


10.62, Reducing the mesh size 

At some stage in the calculation it will usually be necessary, as in 
the above example, to change from a coarse to a finer grid. Let be 
the mesh size of the coarser grid. A convenient first step is to take the 
diagonals of the squares of the old grid as forming a new grid of mesh 
size ^2 = hil\^2 (see Fig. 23). The new grid points are the centres of the 
squares of the old grid. For Poisson’s equation we have on the new grid, 
with the finite-difference approximation adopted 
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and hence 


f},k — i 


(10.29) 
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This gives a set of values of / at the centres of the squares of the old grid, 
which are the grid points of the new grid. In particular for Laplace’s 
equation we have, in this approximation 


fo.k — i 


0 

1 

0 

1 

0 

1 

0 

1 

0 


fj,k^ 


(10.30) 


that is to say, the value of / at the centre of a square is the arithmetic 
mean of its values at the corners. A further relaxation may be carried 
out on this grid, or this process may be repeated immediately, giving a 
grid of mesh size whose grid points are the corners, centres, 

and inid“points of the sides of the original grid (see Fig. 24). 


Example: The example of the previous section continued. Fig. 25 shows the pro¬ 
cess of breaking down the grid in two stages. The numbers in squares are the 
values of V obtained in the calculation shown in Fig. 21. The numbers in circles, 
at the centres of the squares of the original grid, are obtained by the application 
of formula (10.30) to the intermediate grid formed by the diagonals of the original 
grid. The numbers at the other grid points of Fig. 25 are then obtained by the 
application of formula (10.30) to the grid formed by the diagonals of the inter¬ 
mediate grid. 

The residuals are shown on the right of the grid points, and only a single step of 
relaxation is then required to reduce the greatest | jRy | to 2. At this stage another 
significant figure can be taken in V and the relaxation process continued. 
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Note: The advantage of starting with a very coarse grid will now be apparent. The 
number of grid points varies as 1 jh^ and the number of relaxations at each grid 
point probably varies roughly as l//i, so that if the finer grid of Fig. 25 had been 
used from the beginning, something like eight times as much work would be 
required to reach the stage represented by the results in Fig. 25. In terms of the 
analogy explained in note (iii) of the previous section, despite the coarse grid of 
Fig. 21, relaxation on this grid has carried out the bulk of the transfer of charge 
from the inter-electrode space to the electrodes, and what has to be done on the 
finer grid is mainly a minor rearrangement of the residual charges. 
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Fig. 24. 


10.63. Further notes on the relaxation process 

We have only been concerned here with the simplest case in which 
(i) the boundary of the domain of integration does not cut the side of any 
of the grid squares, (ii) the boundary condition is that V is given, and 
(iii) the equation to be solved is the simplest example of an elliptic equa¬ 
tion. For extensions of the procedure to deal with boundaries which 
cut the sides of some of the grid squares, with boundary conditions 
involving the normal derivative of V, and with less simple equations, 
for further practical hints on carrying out the relaxation process in this 
context, and for examples, reference should be made to Southwell’s 
Relaxation Methods in Theoretical Physics and papers referred to in 
the bibliography in that book-t 

f See also E. Stiefel, Zeit, f, angew. Math, und Phys, 3 (1952), 1. 
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The method can be adapted to characteristic value problems such as 
the determination of the values—or at least the smallest value—of A for 
which the equation 


= -A/ 


with boundary conditions / = 0 on a closed curve, has a non-trivial 
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solution. Use of the finite-difference approximation (10.9) on the left- 
hand side gives the set of simultaneous equations 


0 1 0 
1 -(4-AA2) 1 

0 1 0 


fj.k — 0, 


and the determination of A by a relaxation process follows the general 
lines of § 8.73. 


10.64. Richardson-Liebmann process for Laplace’s equation 

There is another process of successive approximation for sohing the 
set of equations (10.10) which form the simplest finite-difference approxi¬ 
mation to . 

^ ^ ^ 0 . 
dx^~' dy^ 


In its simplest form, given by Liebmann,f this process consists of re¬ 
peated use of formula (10.30), working systematically over the grid, 
replacing / at each grid point by the arithmetic mean of the value of / 
at its four nearest neighbours. 

In another form, given by L. F. Richardson,J each value of/^-^. in a 
trial solution is increased by a multiple a of the residual at that 
point, and the result is taken as the next trial solution. Richardson 
proposed the use of a set of different values of a in the construction of 
successive trial solutions. Liebmann’s process is equivalent to a special 
case of Richardson’s in which a is kept fixed. 

Compared with the more recently devised relaxation process, the 
Richardson-Liebmann process has three disadvantages. First, all the 
work is done with large numbers, the values of/themselves, whereas in 
the relaxation process the bulk of the work is done with relatively small 
and simple numbers, the relaxations of / and the residuals. Secondly, 
a lot of time and work is spent on calculation in regions where the 
residuals are small, whereas in the relaxation process attention is first 
directed to the region where the residuals are large and the rest of the 
domain is left untouched until the larger residuals have been removed. 
And, thirdly, it is not so easy to modify so as to take into account the 
higher differences in the replacement of derivatives by finite differences. 


10.7. Parabolic equations 

Most work on the numerical solution of paraboMc partial differential 

equations has been concerned with the equation of heat conduction or 

t H. Liebmann, Sitzungsber, Bayer, Akad. Munchen (1918), 385. 

X L. F. Richardson, Phil, Trans, Roy, Soc, A, 210 (1910), 307. 
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diffusion.I Some practical methods are indicated in the following 
sections (§§ 10.71-10.73). 

As a simple case of a parabolic equation we will consider the equation 
of heat conduction in one dimension 


dl^d^ 

dt 


(10.31) 


with given initial and terminal conditions. 

One way of dealing with this equation is first to replace only one of 
the derivatives by a finite difference; this replaces the partial differential 
equation by a set of ordinary equations which can then be treated by one 
of the methods of Chapter VII. The form of this set of ordinary equa¬ 
tions and the process for their solution differ considerably according as it 
is the first-order (time) derivative or the second-order (space) derivative 
which is replaced by a finite difference. 


10.71. Replacement of the second-order (space) derivative by a 
finite difference 

Let/j(<) be written for the value of / at x = j 8x and at time t. Then 
replacement of the second derivative by a finite difference gives 

^ = [/,«(*)-2/,(()+/;-i(«)]/(8®)HO(8l)'. (10.32) 

This is a set of simultaneous first-order equations for the different func¬ 
tions fj{t) and these can be solved numerically without difficulty. The 
initial value of each/,- is given by the initial conditions. The truncation 
errors are of order (Sx)^; they can be estimated, and the leading term 
in the corrections applied, by Richardson’s ^^-extrapolation process 
(see § 7.51). This method is not restricted to one space variable and the 
time variable, and it is practicable to use it for the numerical solution 
of the equation of heat conduction in two space variables and, moreover, 
for a substance of which the thermal properties vary with temperature. J 

10.72. Replacement of the first-order (time) derivative by a 
finite difference 

For a time interval St, the time derivative at any value of x can be 
replaced by a finite difference as follows: 

iT) = [M'+S«)-/(^.')]/(8<)+0(5()*. 

I For a survey of applications to the diffusion equation, see J. Crank, The Mathematics 
of Diffusion (Clarendon Press, 1955). 

J See N. R. Eyres and others, Phil. Trans. Roy. Soc. 240 (1946), 1. 



FUNCTIONS OF TWO OR MORE VARIABLES 


255 


and, with an error term of the same order, d^fjdx^ at time f-f can be 
replaced by the arithmetic mean of its values at the beginning and end 
of the time interval: 

If the right-hand sides of these are equated and the error terms neglected, 
we have 

= (2/8f)[/(a:,f+S0+/(^,0]-(4/8f)Mi). 

(10.33) 

Given/as a function of x at time t, this is an ordinary differential equation 
for / as a function of x at time There is a set of equations (10.33), 

one for each time interval. But they can be integrated successively, 
and do not have to be treated simultaneously as equations (10.32) do; 
the calculation proceeds interval by interval in t, the results J{x, f-l-8f) 
for the end of one interval being the given function f{x, t) for the begin¬ 
ning of the next. 

In the integration of equation (10.33) it is not necessary to know the 
values of d^fjdx^ at the beginning of the interval; the best procedure is 
to carry out the numerical solution regarding equation (10.33) as an 
equation for the quantity 

u = [f{x,t+U)+f{x,t)] (10.34) 

and then to subtract the known f{x,t) to give f(x,t-\-8t). If we write 
(2/8f) = P, equation (10.33) becomes 

%^-Tc^u = -2kmx,t). (10.35) 

dx^ 

If two separate integrations covering the same range in t are carried 
out, with different time intervals ht, the leading term in the truncation 
error can be eliminated by Richardson’s ^^-extrapolation process (see 
§7.51), and in many cases this will also correct for the next term in the 
truncation error, f 

In this method we carry out successive integrations of a single equation 
(10.35) instead of simultaneous integrations of a set of equations, and, 
moreover, equation (10.35) is a second-order equation with the first 
derivative absent, which as mentioned in § 7.2 is the most convenient 
form of all for numerical treatment. However, the solution of this 
equation has to satisfy two-point boundary conditions in x, and a 

f See D. R. Hartree and J. R. Womersley, Proc, Roy. Soc. A, 161 (1937), 363. 
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step-by-step integration of this equation as it stands may be difficult 
because of the extreme sensitiveness of the solution to initial conditions 
and to rounding errors, which is the more marked the smaller the value 
taken for the time interval Bt. For this reason a more practical way of 
evaluating a solution is the process given in § 7.63 involving factorization 
of the operator {d^jdx^—k^) in equation (10.35), or the matrix factoriza¬ 
tion procedure of Thomas and Fox (§ 8.6).t 

For parabolic equations which are less simple than the simple diffusion 
equation (10.31), and particularly for non-linear equations, the two- 
point character of the boundary conditions and the sensitiveness of the 
direct step-by-step solution makes the method less straightforward than 
it may appear at first sight, and other precautions may be necessary in 
using finite-difference approximations to derivatives.^ 

10.73. Replacement of both derivatives by finite differences 

In the notation of § 10.71, 

1) = [/,((+8()-/,((-8()]/2S(+0(8()» 

These approximations invite us to equate the right-hand sides and so 
obtain (neglecting the error terms) 

/,((+8() = /,{<-S()-{28(/{Sa:)')[/,„(()-2/,(()+/,■-,(()]. 

This looks a very attractive formula, since if the solution has been 
carried to any value of t, it gives directly each value ofseparately 
in terms of known quantities, and a process of using this formula to 
integrate through successive intervals 8t looks simple and straight¬ 
forward. Unfortunately, however, such a process is unstable, and effects 
of rounding errors build up rapidly and uncontrollably.§ 

However, there is another way of using similar approximations which 
leads to a stable numerical process which is practicable but not quite 
so simple.§ This is based on equating approximations to df/dt and 
d^fjdx^ not at grid points in the {x, t) plane but at points half-way in 
t between grid points. If in equation (10.33), d^f{x, t)ldx^ is replaced by 

f For an extension of this procedure to the treatment of a non-linear parabolic equa¬ 
tion, see D. F. C. Leigh, Proc, Gamb. PhiL Soc, 51 (1955), 320. 

X For examples and further discussion, see D. R. Hartree, Rep. and Mem. A.R.C. Nos. 
2426, 2427 (1939, issued 1949). 

§ See J. Crank and P. Nicolson, Proc. Gamh. PhiL Soc. 43 (1947), 60. 
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[/j‘+i(0—2/j-(0+/j-i(0]/(S^)^> S'lid a similar replacement is made for 
d^f{x,t-\-8t)ldx^, the result is 

/,.+,((+S()-{2+(8x)V(S()}/,<(+S()+/,--,(i+5() 

= -[/i«(0-{2-(8*)78(}/,-(()+/i-i(<)]- (10.36) 

This is a set of simultaneous algebraic equations for/y(<-l-8<) as a function 
of Xj with boundary conditions of the two-point type in Xp they can 
be solved by an apphcation of the relaxation process or by some other 
process of successive approximation. 

10.74. Note on methods for parabolic equations 

In all three of the methods considered in §§ 10.71 to 10.73 the pro¬ 
cess of evaluating an approximate solution is carried out in the direction 
of t increasing, t being in the conduction equation (10.31) the time 
variable, and in general that independent variable which does not occur 
in any second derivatives. All three methods are practicable only if the 
domain of integration is open in the direction of this variable, so that the 
whole solution does not have to satisfy any conditions at some later time 
in the course of the process of solution. As already mentioned in § 10.5 
this is the most common situation with parabolic equations. 


10.8. Hyperbolic equations. Characteristics 

For hyperbolic equations methods similar to those for parabolic equations can 
be used, and in addition there is another class of 
methods peculiar to hyperbolic equations. These 
depend on the properties of sets of curves called 
‘characteristics’ of a hyperbolic equation. As in 
§ 10.6, let the equation be 


8x8y 


= 0 , 


(10.37) 


where H, K, L, and M may be functions of any one 
or more of x, y, /, dfjdx, dfjdy, and consider the in¬ 
tegration of the equation along a curve C in the 
{Xy y) plane. Let ds denote an element of arc of the 

curve Cy and djds a rate of change along C (see Fig. 2G). Then for an element of 
arc da of C 
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On substitution from the differential equation (10.37) this becomes 

dx dy 
ds ds 


H^di^AA-L — di^A = 

^ ds W/ ds \8y/ L ds ds 8x8y\ \ds} 


■2K 




ds. 
(10.38) 
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If now the curve C is chosen so that 





(10.39) 

then 

ds ds \dxl 


(10.40) 

Unless dxjds = 

0, these equations can be written 




= 0, 

dx! dx 

(10.41) 


r.dy d i 
dxdx' 


(10.42) 


A curve in the {x^y) plane such that equation (10.39) is satisfied at each point 
of it is called a characteristic. If > HL (and only then), the roots of equation 
(10.39) at any point (x, y) are real and different, so that the characteristics are real; 
it is for this reason that the sign of K^—HL is taken as the defining property to 
distinguish the classes of‘elliptic’, ‘parabolic’, and ‘hyperbolic’ equations. Since 
for hyperbolic equations the roots of (10.39) are distinct it follows that through 
each point of the (x,y) plane there pass two characteristics. Thus there are two 
sets of characteristics covering the (x,y) plane, one member of each set passing 
through each point {Xyy), These two sets will be called ‘set 1’ and ‘set 2’, 

If Hy Ky and L do not depend on /, dfjdxy or dfjdy (though they may depend on 
(^>2/))> characteristics are independent of the particular solution, and can be 
evaluated over the whole relevant domain of the {Xyy) plane before the evaluation 
of a solution is started. But when one or more of Hy Ky and L depend on /, dfjdXy 
or dfjdyy the characteristics depend on the solution and the evaluation of the 
characteristics has to proceed simultaneously with that of the solution. 

The essential feature of the characteristics, from the point of view of numerical 
integration of the equations, can be seen by comparing equation (10.40) with 
equation (10.38). In integration along any curve C, the integrand is a rate of 
change along C. Equation (10.40), as a first-order equation relating dfjdx and 
dfjdyy involves only derivatives in this direction in the {Xyy) plane; hence evaluation 
of the integrand for integration along a characteristic docs not involve any 
differentiation in a direction across that in which the integration is being carried. 
On the other hand the presence of the term in d'^ffdxdy in equation (10.38) implies 
that in integration along a curve G other than a characteristic, the evaluation of 
the integrand for integration along C would involve differentiation in a cross 
direction. We have seen (§ 6.7) that numerical differentiation is a process which 
it is as well to avoid if possible; in the numerical solution of hyperbolic equations 
by integrating along curves in the {Xyy) plane, it can be avoided if and only if 
these curves are taken to be the characteristics. 

It is convenient to write ^Lti, for the roots dyjdx of (10.41), the value of 
at any point referring to the characteristic of set 1 through that point, and the 
value of 1^2 to the characteristic of set 2. Then 

fii+fi2 = HljHy 11^112 = LfH, 

On a characteristic of set 1 we have 
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and from (10.42) — ( 

dx \ 


(10.44) 

on a characteristic of set 2, 

dy 

dx ~ 

(10.45) 

and ,1/ 

dx ' 


(10.46) 


It sometimes happens that the derivatives dfjdx and dfjdyy rather than / itself, 
are the quantities required in the solution, and further that if, if, i, M do not 
depend on /, though they may depend on dfjdy and dfjdx; this is the case, for 
example, if / is the velocity potential of a steady isentropic irrotational flow of a 
compressible fluid, when dfjdx and dfjdyy the components of the velocity, are the 
quantities really required. Then it is convenient to write ity v for dfjdXy dfjdy 
respectively, and (10.44), (10.46) become 


du dv 
dx^^^dx 


on a characteristic of set 1, and 

du 


dv 


dx'^^^ dx 


-{MjH) 


-(MjH) 


(10.47) 


(10.48) 


on a characteristic of set 2. 


10.81. Finite differences between characteristics 

One way of adapting these equations for numerical work is, in effect, to use 
members of the two sets of characteristics as defining a finite-difference grid in the 
{Xyy) plane and to work in terms of finite differences between neighbouring charac¬ 
teristics, f In Fig. 27 the two sets of curves represent the two sets of characteristics; 
the heavy portions represent the portions on which the solution has been carried 
out, arid we want to determine the solution on the set of intersections of which A 
is typical. 

On the characteristic AB of set 1, a finite-difference approximation to (10.43) is 


(10.49) 


Va^Vb = H'IbK^a—^b) \ 

and similarly on AC yA — Vc^ UH' 2 A+H' 2 b)(^a — ^c) ) 

Also on AB a finite-difference approximation to (10.47) is 

(^^—'W^) +J(/X2^+/X22?)(^^“^b) = —^[{^!^i)A^(^^/^)B](^A~^B)> (10.50) 
and similarly on A C, 

{Uji — '^c)~^i(f^lA’^H'lc)i'^A — ^c) ~ ~i[{^j^^)A'^(^/^)c](^A~^c)‘ (10.51) 

The quantities Uy Vy x, y being known at B and (7, this is a set of four equations 
for Uy Vy Xy y sA A. 

l^Hy Ky Ly and so and /X 2 , are independent of/, dfjdXy and dfjdyy then the first 
two equations give the position of A independently of the particular solution, and 
further, the coefficients on the left-hand sides of the second two equations arc 

I See, for example, L. H. Thomas, Commun, on Pure and Applied Mathematics^ 
7 (1954), 159, 
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known in advance of the solutions of these equations. The evaluation of a solution 
is then relatively simple. But if one or more of H, K, and L depend on/, dfjdx, or 
dfjdyy these four equations have to be solved as a set of simultaneous equations for 
Va^ ^re non-linear and can only be solved by trial and successive 
approximation. This makes the evaluation of a solution of the partial differential 
equation in such a case a long and often troublesome and tedious process. 



10.82, Use of given intervals in one independent variable 

An alternative procedure, which has advantages in some contexts, is to use the 
relations (10.49), (10.50), and (10.51) which apply to characteristics, without using 
a grid of characteristics. 

Suppose the solution is known on x = Xq, and consider a given interval Sx to 
X = Xq + Sx, For any point A on the line x = a^o + Sx, let B and O be the points 
at which the characteristics through A cut the line x = Xq (see Fig. 28). Then 
the relations (10.49), (10.50), and (10.51) of § 10.81 apply between the points 
Ay By and (7, but what quantities are known and what quantities are unknown are 
now different. In § 10.81 the points B and C are given and the unknowns are 
XAi yAy '^A> '^Ay point A is given and the unknowns are 11^, '^a* Vb* 

and yc* 

The fact that hx is known, and, moreover, is the same for all points A on the line 
X = Xq + Sx, can be used to simplify the solution of equations (10.49)~(10.51). The 
following is one procedure for finding this solution for a given point A» Equations 
(10.49) can be written 

VA^iH'lA^^ = (10.52) 

Va — (10.53) 

Now suppose y-h^y'i^^y y-h^y^z^^y yn y 2 i^lH) tabulated as functions of y 
for X = Xq, For given 1 /^, estimate u^y these give an approximation to yi^y 
and hence to (y + iyiSx);^ by (10.52). From the table of y + ^yiSx as a function 
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of y, this gives {^l^)By the values of 2 /^, and Vj^ give 

{MIH)j^, So (10.50) gives one relation between Uj^ and Vj^. Similarly ( 10 . 53 ) gives 
an approximation to 2 /< 7 , and (10.51) then gives another relation between and 
Vj^, These two relations can then be solved for Uj^ and Vj ^. If the estimates have been 
correctly made, the values of and Vj^ so calculated will reproduce the estimates. 
One way of achieving this is to take three sets of estimates, namely = (a, 6 ), 



Fig. 28 . 


(a+8a, 6), and (a, 6 + 86) and use linear interpolation to find the values of and 
for which estimates = {a-\-p-^^a,b-\-p 2 ,^b) give results in agreement 

with the estimates; the calculation for this point A should be repeated with these 
estimates as a check. If the first estimate (w^, Vj^) == (a, 6 ) is not good, this process 
may have to be repeated. 

This process may be simpler than the solution of equations (10.49)~(10.51) for 
Va* '^a> '^Ay given B and (7. Fiirther it may give the results in a more 
convenient form, particularly if x is physically a time variable, for it gives results 
directly at exact values of a:, which is the form in which they are likely to be 
required, whereas if results are obtained on a grid of characteristics, extensive 
interpolation is required to obtain a set of results for a set of values of x. Also the 
intervals at which results are obtained are completely under the control of the 
individual who is doing the work, instead of being determined by the shapes of 
the characteristics as the solution proceeds. 

10.83. Two simultaneous first-order equations 

For two simultaneous first-order equations, linear in the first derivatives, there 
may also be two sets of characteristic curves such that the evaluation of the 
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integrand along these curves involves no differentiation across them. Let the 


dx'^^^dy'^^^dy ' 




(10.65) 


in which H^, H^, K^, K^, M^, and niay be functions of one or more of the variables 

x,y,f,a.ndg. 

To find curves with the property required, multiply equation (10.55) by A (which 
may be a function of x and y) and add equation (10.54). The result is 

[^+(H,+XK,)^]f+x[^+{H,+X-^K,)^]g = ikf,+AAf,. (10.66) 


(10.67) 


In order that the operators 

and + 

should represent rates of change in the same direction in the {x,y) plane, it is 
necessary that H^+X-^K^ = H^+XK^, 

that is, X^K 2 ~\-X{H^—H 2 )—Ki = 0, 

If the roots A^, A 2 of this equation are real and distinct, there are two such directions, 
given by ^ Hi+XiK^, dyjdx = iZi+AjK^. (10.58) 

The pair of equations (10.54), (10.55) is then called ‘hyperbolic’ and the sets of 
curves on which dyjdx is given by the values (10.58) are called ‘characteristics’ 
of these equations (they are the characteristics, in the sense of § 10.8, of a second- 
order equation which can be derived from the two first-order equations). 

For any function of x and y, say h(Xyy)y [5/5x + (£ri+AijK’i)5/^2/]fc is the rate of 
change of h with x along a curve C for which dyjdx == i/i-fAijFTg. Hence on a 


curve (7i, given by 
equation (10.56) becomes 


(dyldx)i = Hi+XiK^, 


and on a curve Cj given by 
equation (10.56) becomes 


(dyldx)z = ffi+AjiTj 

{dg' 




(10.69) 


(10.60) 


(10.61) 


(10.62) 


These equations are very similar in form to equations (10.43) to (10.46) of § 10.8, 
and can be treated in a similar way. One important application is to the non¬ 
steady motion of a compressible fluid; in this case x is physically a time variable and 
y a space variable, and the roots of equation (10.57) are always real. 

The relation between the characteristics of the two first-order equations (10.54) 
and (10.55), and those of a second-order equation can be shown as follows. 
Equation (10.57) expressed in terms of the values of dyjdx = £ri-bA^r 2 on the 
characteristic, is \2 ^ 

-(H,+H2) J+h,H2-j?,j?2 - 0. 


(10.63) 
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Also differentiation of equation (10.55) with respect to and of equation (10.54) 
with respect to x and to y, and elimination of second derivatives of g between the 
results, yields an equation in which the terms involving second derivatives are 

which are the second-order terms in an equation (10.37) in which 
il=l, = L = H,H,-K,K,; 

and for this differential equation, the equation (10.42) for the characteristics in 
the sense of § 10.8 is just equation (10.63). 
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MISCELLANEOUS PROCESSES 

11.1. Summation of series 

In practical applications of numerical analysis, as distinct from artificial 
examples constructed for the purpose, it is comparatively seldom that 
the original formulation of a problem is the summation of a series, though 
summation of a series is sometimes a useful method of deaUng with a 
problem originally formulated in some other terms. 

For example, the properties of the Airy function Ai(a;) which make it 
important in apphcations are these: 

(i) it is a solution of y" = xy which tends to zero as x tends to infinity; 

this defines it except for a constant multipl 3 dng factor; 

00 

(ii) it is I cos(a;<-|-i^^) dt. 

0 

It can be evaluated from either of these properties without the use of its 
expansion as a power series in x. This power series expansion is a further 
property which happens to be useful in the evaluation of Ai(a:) for small 
values of x, but it is not the primary reason for the importance of this 
function, nor a property which need be used at all in its evaluation. 

A series is useful in numerical work only if the sum of the first few 
terms is an adequate approximation to the sum of the series, or to the 
function represented by the series—just what a ‘few’ terms and an 
‘adequate approximation’ mean will depend on the context. Suppose 
we have a numerical problem originally formulated in some other way 
than the summation of a series, and in trying to evaluate results by 
summing a series we find that the convergence of the first few terms is 
not rapid enough for them to be useful. Then this is a strong hint that 
evaluation of the series is not the best process for getting the results 
required, and the possibiUties of other processes should be investigated. 

But sometimes we may be concerned with the summation of slowly 
convergent series either in calculations originally formulated in such 
terms, or through the reduction of a more comphcated situation to such 
a summation. In such cases we need processes for transforming slowly 
convergent series into more rapidly convergent ones. The simplest such 
transformation is one due to Euler for a series of terms of alternating 
signs. 
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11.11. Euler’s transformation for a slowly convergent series of 
terms of alternate signs 

This transformation can be derived by an application of finite-differ¬ 
ence operators, and is one of the few cases in which the use of forward 
differences gives the most convenient form for results. 

Let the magnitudes of the terms be u^, in general so that 

the series which we wish to sum is 

S = = 2 ( —(11.1) 

n 

Let us take the successive differences of the terms UJ^, regarded as a 
function of n. Also let A be the forward-difference operator with respect 
to 71, defined by Au^ = and E = 1-f A. Then and 

S = {i — E+E^-E^+...)Uf, = ^ 

= iK—(1L2) 

The differences involved here are the fonvard differences from the first 
entry of the table of u^, and are all available. 

If the series (11.1) is slowly convergent, then the successive differences 
of the Uf^s usually decrease rapidly and the series (11.2) converges 
much more rapidly than the series (H.l). It will often be best not to 
carry the transformation back to the beginning of the series to be 
evaluated, but to calculate separately the sum of the first N terms 
{N = 6 OT 8, perhaps) and apply the Euler transformation to the re¬ 
mainder. A good check on the results can be obtained by carrying out 
this process with two different values of N. 


Example: To calculate S(x) = 2 (~ ^ 

7n = 0 

A table of the function 107(a:+m)2 for x — 10, m = 0(1)10 and its differences 
up to the sixth order is given on p. 266; the effects of rounding errors are becoming 
marked in the sixth differences. The value of S{x) is the sum of alternate first 
differences of — 1 /(a;+ 7 ^ 1 ) 2 . If wc take the first of these first differences (that is, the 
first two terms of the series) and apply the Euler transformation to the remainder 
of the series, we obtain 

107^(10) = 17355 + i[69444 + J(10272) + i(2120)+i(544) + fo(162) + 


= 17355+37595 = 54950. 


+ A(52) + e+(14) + ...] 


The values of differences used are those underlined in the table. 
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m 107 (a;+w)* 

0 100000 




-17355 






1 

82645 

-13201 

4154 

— 1225 




2 

69444 

-10272 

2929 

-809 

416 

— 151 


3 

59172 

-8152 

2120 

-544 

265 

-103 

48 

4 

51020 


1576 


162 


51 



-6576 


-382 


— 52 


5 

44444 


1194 


110 


14 



-5382 


-272 


-38 


6 

39062 

— 4460 

922 

-200 

72 

-19 

19 

7 

34602 

— 3738 

722 

— 147 

53 

-19 

0 

8 

30864 

— 3163 

575 

-113 

34 



9 

27701 

-2701 

462 






10 25000 

If we take the first foiir terms of the series and apply the Euler transformation 
to the remainder, we obtain 

10’5(10) = 17355+10272 + 

+ i[51020+i(6576)+i(1194)+|(272)+*(72)+A(19) + ...] 

= 27627 + 27323 = 54950. 

This agrees with the value already calculated, and we obtain the result 
5(10) = 0-005495 to six decimals. 


If the ratios of successive terms are nearly constant, a modified 

form of the Euler transformation can be used effectively. Let 

jS being a number chosen so that the variation of with n is small. Then 


1 


j_ 

iS+l 


^ 




^ 






AX+...j. 


(11.3) 


11.12. Use of the Euler-Maclaurin integration formula in the 
summation of series 

When f{x) is a function such that J f{x) dx can be integrated formally, 
the Euler-Maclaurin formula (6.22) can often be used effectively for 
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evaluating sums of the type 2/(m) over a set of integral values of m. 

m 

From formula (6.22) with a;,, = 0 and interval (So;) = 1, we have 

/o+/l + *--+/n 


Xn 

= / /(.r) dx+\u,+h)+US'n-S«)-MJ'i-f:)+A>(n-!l)---- 


(11.4) 


and in particular, if/(a:) and all its derivatives tend to 0 as a: tends to oo, 

oo °° 

~ “S5I15/0 + — (11.5) 


As in the previous section, the result of using this formula can be checked 
by appljdng it to the series formed by omitting the first few terms from 
the series to be summed. 


Example: To evaluate ^ l/{8+w)*. 

m — 0 

CO 

For /(a;) = l/(o+a;)*, we have J f(x) dx = 1/a, 

0 

and /'(O) = —2/a*, /"'(O) = —24/a*, /TO) = --120la\ 

so evaluation of formula (11.5) for a = 8 gives 


V 1 //Q 1 1 24 , 

8*^2 64*^12 8* 720 8*"^“' 


= •125 + -007ai 25 +-00032 65--00000 10+... 


= *13313 7 to six decimals. 


Also 


GO 


CO 


2 l/(8+m)* = - h +- h + 1 l/(10+m)* 

m«0 m*=0 


and evaluation of formula (11.5) for a = 10 gives 


00 111121 24 
^2ol/(10+w)* = —+ - j^+Y2 

= •1 + -005 + -00016 67--00000 03 
= -10516 64 

f l/(8+w)* = -01562 6+-01234 57 + -1051664 

m = 0 

= *13313 7 to six decimals 


verifying the value obtained by evaluating formula (11.5) with a = 8. 


Slowly convergent series of positive terms which cannot be handled 
by this application of the Euler-Maclaurin formula are often difficult to 
deal with numerically. If the terms are given by an algebraical formula, 
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then it may be possible to find an analytical transformation which con¬ 
verts the series into a more rapidly convergent one, but this procedure 
is not usually available unless each term is of a relatively simple form.f 

11.2. Harmonic analysis 

Harmonic analysis is concerned with the representation of a function 
f{x), over a finite range of x which will be taken as 27 r, as a series of 
circular functions of x: 

f{x) — ^^0-}-^iCosxH-u42Cos2cc-f-...+-SiSinx-f--B2sin2a;+... . 

( 11 . 6 ) 

The most important applications are to cases in which f{x) is periodic 
in X with period 277, or in which f{x), although not periodic, or not 
defined outside a range a^Q ^ a: ^ (xQ-f 277 ), satisfies the conditions 

/<B(x„+2,r)=/(»(a:„). (11.7) 

If f{x) does not satisfy such conditions, or if it has discontinuities in 
magnitude or in a differential coefficient of low order, then harmonic 
analysis is usually of formal rather than numerical interest, since in 
numerical work only a finite number of coefficients in the series (11.6) can 
be evaluated, and the sum of any finite number of terms gives a function 
which satisfies the conditions (11.7) and has no discontinuity in any 
derivative. If / itself has a discontinuity or does not satisfy 

/(^o+ 277) =/(cCo), 

then the behaviour of the sum of a finite number of terms in the neigh¬ 
bourhood of the discontinuity (or of cCq and X(^-\- 27 t) differs considerably 
from the behaviour of/(cc); as n increases, the behaviour of / remains 
of the character shown in Fig. 29, the scale of x, but not the scale of the 
oscillations in /, becoming smaller as n increases. This is known as the 
‘Gibbs phenomenon’ and illustrates the need for caution in regarding 
a finite number of terms of the series (11.6) as an adequate representa¬ 
tion of the function unless it is free from discontinuities and the conditions 
(11.7) are satisfied. 

The most usual applications of harmonic analysis are in connexion 
with the analysis of results of experiment or observation. Occasionally, 
however, it may be required in purely analytical or numerical contexts. 

f For other methods of treatment of such series, see J. C. P. Miller and W. G, Bickley, 
PhiL Mag, (7) 22 (1936), 754; T. M. Cherry, Proc, Camh, Phil, Soc, 46 (1950) 436; G, G. 
Macfarlane, Phil, Mag. (7) 40 (1949), 188. See also O. Szasz, Journ, Math, and Phys, 28 
(1949), 272. 
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For example, in the solution of Laplace’s equation in two dimensions, 

^ .a2F _ 

dif~ ’ 

in the interior of the unit circle, it may be convenient to use the result 
that V can be written 

V = f^+r(^icos04-5isin0)4-r2(^2cos204-52sii^20)-}-... . 
Harmonic analysis of F as a function of 6 on the unit circle gives Vq and 
the coefficients and directly, and hence the solution F, without 
requiring numerical integration over the whole interior of the unit circle. 



Fig. 29. 

The coefficients in the series (11.6) are given by 

27T 27T 

7tA^ = J f{x)eosnxdx, 77 = J f{x)smnxdx. (11-8) 
0 0 

li f{x) satisfies the conditions (11.7) it follows that, in each of these 
integrals, each derivative of the integrand has the same value at the 
upper limit as at the lower limit. Hence in the Euler-Maclaurin formula 
(6.22) for each of the integrals the correcting terms from the two ends 
of the range cancel identically, and if the integrals can be evaluated 
from a set of values of f{x) at equal intervals on x, the appropriate 
integration formula is the trapezium rule without corrections.! Hence 
if the range 2tt in x is divided into K equal intervals, we have 

hKA^= ^ fix^)eos7ix^, hKB,^= 2 f{x^)sinnx^, (11.9) 

k = Q A: = 0 

where X/^ = ^TrkjK. 

t See, however, the discussion of integrals of this kind in § 6.54. 
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These expressions for the coefficients are not significant for n > \K. 
This can be seen as follows. Let be a value of n less than \K, and 
m any positive integer. Then at the points = 2TTklK we have 

cos{mK:iz'>^o)^k = cos{2TTmk:iz'>^o^k) — cosWo^^j., 

sm{mK:^nf^)X/^ = sin{2TTmk:iz'>^o^k) — rhsinWoX^j.. 

Hence at these points the contributions from the terms with 
n = mKz^riQ have exactly the same variation with k as contributions 
from the term with n = n^, and no analysis using only the values of / 
at these points can distinguish the contributions from the values 
n — with different values of m. The first of the sums (11.9) 

gives the same values for each (n = values of the 

.4^’s are independent, so these values given by (11.9) cannot all be signifi¬ 
cant. 

The point is that for values of n greater than jK the values of x,^ are 
not closely enough spaced, relative to the period of sin nx, for the formulae 
(11.9) for the integrals to be valid. For n — \K the values of the inte¬ 
grand /(x)cos nx at successive values of Xj^ are 

and for most functions / these values are too irregular to give any 
confidence that they represent the behaviour of the integrand well enough 
to justify any numerical work on it at all. Their differences diverge 
and the situation is similar to that considered in § 6.54, where also w'e 
were concerned with an integral for which the correction to the trapezium 
rule vanished at both ends of the range of integration, but an incorreet 
value was obtained if too great an interval of integration was taken. To 
define an oscillating function adequately it is advisable to have at least 
six points per period, and this suggests that the series (11.9) should not 
be regarded as adequate approximations to the integrals (11.8) for 
n>iK. 

Another aspect of these results can be illustrated by considering an 
alternative way in which the coefficients might be determined 

numerically, not as values of the integrals (11.8) but by fitting the series 
(11.6) to f{x) at the discrete set of values X/. of x. If this is done, then we 
have 

K-l N 

k = 0 

2f{Xk)sinnX^ = W^n-^A'-n+^A'+,i--S2A-n + -] 

A: = 0 / 


. ( 11 . 10 ) 
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It is clear that with the restriction to K values of x, it is not possible to 
determine more than K relations between the coefficients, namely the 
first of relations (11.9) for n. = 0 to \K and the second for w = 1 to 
{\K —1); the terms 5„sin7ia:j;. for n = {m-\-\)K make no contribution 
to the sum (11.6) at any of the points x = Xj^. The smoothest function 
with the assigned values of/(a;;^.) will be that for which = 0 

for n > \K, and then for n ^ \K are given by (11.9). 

If/(a*) is a continuous function, then a good test of the significance 
of the values of A^, calculated from (11.9) is given by making two 
analyses with values of K which are relatively prime or have only 
a small common factor, such as K = 30 and 32, or 48 and 50; this pro¬ 
cess also provides a good overall check on the results. 

11.3. Recurrence relations for a sequence of functions 

The Bessel functions of integral order JJix) form an example of a set 
of functions of one variable {x) and one parameter {n) which have a 
number of properties in common, such as the form of the differential 
equation satisfied by them and their asymptotic behaviour. They are 
connected by relations between the functions of different orders n, 

such as (11-11) 

= (inlx)J„(x). ( 11 . 12 ) 

Such relations are called recurrence relations. Other examples of such 
sets of function are the Legendre functions P„(a:), the confluent hyper¬ 
geometric functions W^„,(a:) of Whittaker,f and the Weber functions 

^n(^)-t 

It is often convenient to use such recurrence relations to evaluate 
functions, for some value of the parameter for which there may be no 
tables available, from tabulated values of the functions for other para¬ 
meter values. Such a process must be used with care or it may lead to 
quite spurious results. This can be seen by considering, as an example, 
the evaluation of J„(a:) for a given value of x and for large values of n 
from Jo{x) and Ji{x) by repeated use of the relation (11.12). 

For Jn{x) we require that solution of (11.11) which tends to zero as 
n tends to infinity. But if w'e evaluate J^ix), Jn+i{x), in succes¬ 

sion by using (11.12) in the form 

J^^,{x) = {2nlx)JJx)-J„_,{x), (11.13) 

the rounding errors introduce a small multiple of the second solution 

f E. T. Whittaker and G. N. Watson, Modem Analysis (C.U.P. 1927), ch. 16. 

{ Ibid., § 16.2. 
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Y^{x) of this recurrence relation. For n <. x this remains small, but 
for n > a; it behaves roughly as an increasing exponential, and increases 
without limit as n tends to infinity. 

Thus this way of using the recurrence relation is not satisfactory for 
calculating Bessel functions for n > x, though it is satisfactory 
for n < X. It would, however, be satisfactory for calculating YJ^x), 
since in this case the unwanted solution, of which a small multiple may 
be introduced by rounding errors, is one which decreases indefinitely, 
relative to the wanted solution Y^^{x), as n increases. 

On the other hand, the range over which (n/x) > 1 is just the range 
over which relaxation methods can be used effectively for the solution 
of (11.12), provided the solution has to satisfy two-point boundary 
conditions in n. This is the case for the Bessel function since up 

ton = X these can be built up satisfactorily by successive use of formula 
(11.13). This gives J^in) as one terminal condition in n, and the other 
is given by Jn{x) 0 as w -> oo. 

This example shows how quite different results can be obtained by 
different ways of using the same simple formula; one way of using the 
recurrence relation (11.12) may lead to quite spurious results although 
no mistakes have been made in the calculation, whereas another way of 
using the same formula can be used to give results accurate to any 
assigned degree. 

11.4. Smoothing 

‘Smoothness’, either of a continuous function or of a set of discrete 
values, is a property of which it is difficult to give a quantitative defini¬ 
tion. For a continuous function it implies smallness of high-order 
derivatives, and for a table of function values it implies smallness of 
the higher orders of differences; this implies also regularity of the differ¬ 
ences, since if the wth differences are irregular, the (w-|- 10)th differences 
will not be small. 

By ‘smoothing’ a set of function values is meant a process of replacing 
them by another set which differ only slightly from them but are 
‘smoother’ in this sense. If each member of a set of function values has 
been obtained by an independent calculation, and each is subject to a 
rounding error, then the accuracy of the values may be increased some¬ 
what by a smoothing process. But this improvement should not be 
relied on, and cannot be estimated. It should not be relied on because 
it is always possible that the rounding errors in a number of consecutive 
function values may be of the same sign and similar in magnitude, and 
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then smoothing will not improve them. Further, as we have already 
seen in Chapter IV, an incorrect set of function values may have smooth 
differences, and a set of correctly rounded-off function values may be 
less smooth than a set obtained by rounding off incorrectly. Also 
without knowing in some other way a more accurate set of function 
values, there is no criterion by which the improvement of the function 
values can be assessed; and if these more accurate function values were 
known, there would be no point in carrying out the smoothing process. 
If a set of function values is too much affected by rounding errors, the 
only reliable way of getting more accurate values is to carry out the 
calculation of the function values to greater numerical accuracy. 

The main purpose in carrying out a process of smoothing must there¬ 
fore be to achieve smoothness, not accuracy. The contexts in numerical 
analysis in which smoothness is a prime requirement are not many, so 
that such a process is not often required. But occasionally it is difficult 
to make satisfactory progress without one. 

Consider, for example, the evaluation of a set of solutions of a differen¬ 
tial equation involving a function f{y) determined by experiment or 
by statistical samphng, the different solutions being distinguished by 
different initial conditions or different values of one or more parameters. 
For consistency between the various solutions, and also in order to use 
the differences of intermediate quantities for checking the integrations, 
it may be advisable to use in the numerical work a table of/(y) which is 
smooth to a substantially greater degree of numerical accuracy than the 
accuracy of the experiments from which f{y) is determined. This can 
sometimes be achieved by fitting an analytical formula to the experi¬ 
mentally determined values; when this has been done, the formula can 
be evaluated to any required numerical accuracy. But this process is 
inconvenient unless a relatively simple formula can be found to fit the 
experimental values within the experimental or sampling error, and it 
is also unnecessary. A more purely numerical smoothing process is 
often more useful and more effective. 

Another example is provided by the process of § 10.72 for the integra¬ 
tion of a parabolic partial differential equation. For one time interval 
of this process, as applied to the equation 

8t dx^ ’ 

the equation (10.33) is solved with two-point boundary conditions in x. 
Suppose that in the process of evaluating the solution satisfying the 
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two-point boundary conditions, random rounding errors may occur in 
up to in the last significant figure kept. If the 
rounding error in/(a;, t) may be that in f{x, may be 
that in/(a;, t-\-28t) may be ±(p+2g) and so on, and effects of rounding 
errors may be somewhat increased if Richardson’s process of ^^-extra- 
polation is used to correct approximately for the truncation error. Thus, 
however many figures are kept, the last will become more and more 
irregular as the calculation proceeds. If two or three guarding figures 
are kept, this will not affect the final results significantly, but such 
irregularities make it difficult to use differences for checking, and may 
lead to time being wasted in trying to find a suspected mistake that is not 
there. For this reason it is advisable occasionally to smooth/(x,i) as a 
function of x during the progress of the calculation. 

11.41. Automatic methods of smoothing 

A simple example of one class of methods of smoothing is the following; 
Replace ea;..h function value by the mean of five successive values 
of/centred on fp that is, take 

fj — 5(/j+2+/j+l+/i+/j-l+/j-2)- 

This is sometimes called ‘smoothing by fives’, or ‘smoothing by groups 
of five’. In this process, the irregularities get smoothed out by being 
distributed among neighbouring function values. This can be illustrated 
by the set of function values 

/ 00001000 0, (11.14) 

for which this process gives 

/ 0 0 J J i i J 0 0. (11.15) 

The maximum value of ] 8^f \ is I whereas that of | Sy | is 2: on the basis 
of this criterion, the set of values (11.15) is ten times as smooth as the 
set (11.14). 

This process of smoothing by groups of (2ri+1) can be repeated. For 
example, two successive processes of smoothing by groups of three, 
starting from the set of values (11.14) gives 

I 0 0 ^ I § I 1 0 0. (11.16) 

For this set of values, the maximum 1 8“f \ is |. 

These are two examples of a general method which consists of replacing 
each fj by a linear combination 

fj= 1 (11.17) 

A— —n 

of function values centred on fj. The smoothest set of function values 
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is simply fj = constant, and in order that these should not be altered 
by the smoothing process, the coefficients must satisfy 

i «/. = !; 

and normally the coefficients will be symmetrical about k = 0. Different 
processes are given by different choices of the coefficients in (11.17). 

These methods are often unsatisfactory in practice for three reasons. 
First, once the particular smoothing formula to use has been decided, 
it is automatic in character in that the results are then determinate, and 
gives no opportunity for the exercise of judgement by the individual who 
is carrying out the calculation. This might at first sight seem an advan¬ 
tage, since the results will then be independent of the individual. But 
this apparent definiteness of the results is spurious since there is a good 
deal of latitude in the choice of what smoothing formula to adopt. And 
the smoothing process is in practice one in which it seems desirable to 
give the individual who is carrying it out some discretion on matters 
such as the degree of smoothing at which to aim and the degree to which 
changes/^—/j- from the original function values are acceptable. Secondly, 
with methods depending on the use of formulae of the type (11.17), the 
smoothed values^ cover a smaller range of_; than the original values; 
in a method due to Spencer,! recommended by Whittaker and Robin¬ 
son, J ten values at each end of the range are lost, so that from 30 values of 
fj only 10 smoothed values in the middle of the range of_; are obtained. 
Such a loss of range is often unacceptable. Thirdly, a special procedure 
is needed if it happens that some value of f{z) is known exactly, such as a 
value f{x) = 0 at X = 0, and is not to be modified by the smoothing 
process. 

The dangers of a blind use of an automatic smoothing process are 
illustrated in Fig. 30. Here the full curve is representative of the be¬ 
haviour of the function/(v) = where R is the resistance of the air 
to a body moving through it at a speed of v ft./sec. If f{v) is tabulated at 
intervals of 60 ft./sec. the behaviour of the second and higher differences 
of f{v) is rather violent, and can be molUfied by the appUcation of a 
smoothing process. Spencer’s process, apphed to these data, gives 
results represented by the squares and broken curve in Fig. 30. They 
are certainly smoother (the greatest value of |8y| has been reduced 
from 232 to 48 in terms of the third decimal place as unit). But it does 
not follow that the smoothed values are a better representation of the 

I J. Spencer, J, Inst, Acttcaries, 38 (1904), 334. 

J Calculus of Observations (Blackie, 1940), p. 290. 
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actual behaviour of f{v) than the unsmoothed values; they are almost 
certainly worse, and in particular the minimum about v = ISO ft./sec. 
is almost certainly spurious. But if one insists on using an automatic 
formula one has no control over the results it is going to give; if one 



beUeves the use of the formula to be significant at all, all that one can 
do is to accept the results of using it. 

11.42. Smoothing by use of an auxiliary function 

A less formal but more practical method is due to A. T. Doodson.f 
It is based on the use of graphs. 

Let f{x) be the function which it is desired to smooth. Unless f{x) 
is of only two or three figures, and sometimes even if it is of three figures, 
it will not generally be possible to smooth it directly by plotting and 
drawing a smooth curve ‘through’ the plotted points. But if g{x) is a 
smooth function approximately equal tof{x), it may be possible to plot 
the difference f{x)—g{x) on a scale open enough to smooth it, to the 
degree of numerical accuracy required, by such a graphical process. 
Then/(a:), the smoothed function by which/(.r) is replaced, is constructed 

/(^) = g{x)-\-smoothed{f{x)—g{x)}. 

f This method was devised during the war of 1914-18, in connexion with ballistic 
work, but only published recently, in Quart, J, Mech, and Applied Math, 3 (1950), 217. 
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The auxiliary function g{x) can be formed in several ways. It may, 
for example, be taken to be given by an analytical formula, such as 
ax^, axJ{x^-{-h'^), e^, if there is any theoretical reason or empirical 

indication that/(a:) is approximately of such a form. Another process 
is to build up g{x) from a smooth set of differences. This is of more 
practical use in many cases, as it can be used equally well whether or not 
a good approximation to f{x) can be obtained by a simple analytical 
formula, and it does not involve any selection and adjustment of para¬ 
meters in an anal 3 d.ical formula so as to get a good overall fit to f{x). 

For simplicity suppose/(ic) to be given at equal intervals of x. And as 
an example of the general process, suppose this function, and the 
interval of tabulation, to be such that the range of the values of 8^f is 
not more than 200, so that they can be plotted on such a scale (1 mm. 
or ^ in. to a unit) that they can be read off to a unit. 

The process is then as follows. Plot 8^f{x) and draw ‘through’ the 
plotted points as smooth a curve as possible without smoothing away 
significant features of the behaviour of the second differences. This is 
one place at which discretion is required in judging what features of the 
behaviour of the second differences are significant. If the uncertainty 
of each value oif{x) is known, the range of uncertainty of each second 
difference can be found and indicated on the plot, and this may help in 
distinguishing significant from non-significant features of the variation 
of the second differences. It is better to over-smooth at this stage rather 
than the reverse; significant variations which are smoothed out at this 
stage are replaced at a later stage. 

Let h{x) be these smoothed values of 8^f{x). It is not advisable to 
double-sum them directly to give the auxihary function g{x). A small 
systematic difference between two different ways of drawing the curve 
from which h{x) is read off may build up, on double-summing, to a 
substantial amount, so that though one curve might give a g{x) which 
differed little from/(a;), the other might give a g{x) departing from it to 
such an extent that it would be difficult to plot f{x)—g{x) on an adequate 
scale; and there can be no certainty that the curve from which h{x) is 
read off is not of the latter kind. It is therefore best first to form the single 
sum ah{x) of the values of h{x), and to modify this if necessary so as to 
get a general agreement with the first differences of f{x), before forming 
an auxihary function. The differences 

hf{x)—ah{x) 

between the first differences of/(a:) and the first sum of h{x) are therefore 
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plotted and smoothed graphically. The smoothed values of 

Sf{x)—ah{x) 

are then added to the values of ah{x) to give the first differences of the 
auxiliary function g{x), which is then built up from these diffe^'ences. 
Thus g{x) is given by 

g = a[a^-l-smoothed(8/—a^)]. 

Finally, (f—g) is plotted and smoothed, and the smoothed function 
fix) is given by 

fi^) = 9 '(a;)+smoothed{/(a:)—sr(a:)). 

In this final stage discretion can again be exercised regarding the 
extent to which values of/may be modified by the smoothing process, 
and the significance of various features of the behaviour of f. The 
difference between the original and smoothed values of f at any value 

of a IS ^ {/(a;)—gr(a;)}—smoothed{/(a:)—gr(a:)}, 

and the right-hand side here is the departure of the smooth curve from 
the plotted point (/— g) at each value of x. If the range of uncertainty 
of each value of/, or the maximum change in each value which would be 
acceptable, is known, this can be indicated on the plot, and the smooth 
curve drawn so that its departures from the plotted points do not 
exceed this range at any point. In particular, if at any point the value 
oi f{x) is known exactly, the curve of (f—g) must be drawn to pass 
through the plotted point at that value of x. In Doodson’s method of 
smootliing, particular features such as this can be taken into account 
quite easily and without departure from the regular procedure. 

The differences of the final values of / (x) provide a check of the cal¬ 
culation and an indication of the degree of smoothness which has been 
obtained. An indication of the extent to which the final results depend 
on the details of the process of smoothing is given by carrying out the 
process twice using the same set of values o{f{x) but different smooth 
curves from which to read off the smoothed second differences of h{x). 

The process can be adapted to start from other orders of differences 
than the second. However, if f{x) is at all seriously irregular, the higher 
differences of f{x) probably vary so wildly that it is difficult to see any 
general trend in their values. 

Examples of this process, and its extension to functions of two 
variables, can be found in Doodson’s paper. 


XII 


ORGANIZATION OF CALCULATIONS FOR AN 
AUTOMATIC MACHINE 

12.1. Automatic digital calculating machines 

When we write a number in the ordinary way, such as 1925, the symbols 
such as 1, 9, 2, 5 in this example stand for what we call the digits of a 
number, and a piece of equipment which operates directly with, and 
records, the discrete digits of each number is often called a digital 
calculating machine. Since about 1938 there has been a great develop¬ 
ment of such machines with two important features. First, they can 
carry out long and intricate numerical calculations quite automatically 
once they have been provided with a specification, in a suitable form, 
of the calculation to be carried out. And, secondly, they are very 
versatile, so that the same machine can be used for many quite different 
kinds of calculation; for example, for calculating values of a function 
from its power-series expansion, for solving large systems of linear 
simultaneous equations, for finding the characteristic values of matrices, 
and for the step-by-step integration of ordinary differential equations. 
To express these two features, such machines are sometimes called 
general-purpose, automatic, digital calculating machines. 

The process of organizing calculations for such machines is a branch 
of numerical analysis which has only come into being with the machines, 
and this chapter is included here to give an introduction to the subject. 
It is concerned with the planning of calculations for such machines 
rather than with the machines themselves; it is only concerned with the 
machines in so far as their characteristics affect the process of organizing 
calculations for them. 

To see what is required of such a machine, consider first the organiza¬ 
tion of a calculation carried out by hand with the assistance of a desk 
machine. This is represented diagrammatically in Fig. 31. There are 
three kinds of equipment the computer has to assist him; these are 
represented by rectangular blocks in the figure. One is a desk machine, 
another is a set of tables, and the third is the working sheet on which 
intermediate and final results will be recorded and on which should be 
written enough data to identify the calculation and to summarize the 
calculating procedure. 

When the method for doing a calculation has been decided, the detailed 
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process of carrying it out consists of (i) a sequence of arithmetical opera¬ 
tions carried out on the machine, or perhaps on a slide-rule as auxiliary 
equipment, or mentally in the case of simple operations such as multipli¬ 
cation or division by 2 or addition of pairs of numbers, and (ii) transfer 
of numbers between the three blocks represented in Fig. 31. 


Tables 



The transfers and the arithmetical operations are controlled by the 
individual who is carrying out the calculation, who is represented by the 
large circle in Fig. 31; the controls he exerts are represented by directed 
lines from the controller to small circles representing control of the 
transfer of numbers, and to the desk machine. He also takes from the 
working sheet information about the arithmetical operations to be 
carried out and their sequence; this is represented by the directed line 
from the work sheet to the computer. 

An automatic machine must be capable of carrjdng out the same pro¬ 
cesses, and can be thought of as having a similar organization, as shown 
diagrammatically in Fig. 32. It must have an arithmetical unit in which 
arithmetical operations can be carried out, to take the place of the desk 
machine in a hand calculation; a store both for numbers and for operating 
instructions, to take the place of the work sheet and tables; and a 
control system to take the place of the human computer who controls the 
sequence of operations in a hand calculation. The machine also needs 
input and output equipment for receiving numerical data and operating 
instructions from the outside world and for delivering its results. 

Whatever the physical form of the store, it must provide a number of 
identifiable storage locations, and it is convenient to think of these as 
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distinguished by being numbered. The number which is the label of any 
storage location is often called its ‘address’, or the ‘address’ of its 
content. C{n) will be used for ‘the content of storage location n’. It is 
sometimes convenient to represent the address of the number which is 
the value of a quantity x by L{x) or A(x), or by C~^{x) if it is desired to 
emphasize that the relation n = L{x) is the inverse of the relation 
X = C{n). 



Fig. 32. 


The specification of an operation which the machine is required to 
carry out is called an ‘instruction’ or ‘order’ (sometimes a ‘command’), 
and the ordered set of such instructions needed to carry out a calculation 
is called the ‘program’ for that calculation. An important feature of 
most machines is that instructions are coded in such a way that they 
have the same form, within the machine, as numbers, the difference 
between numbers and instructions being in the way they are used. The 
content of a storage location is then usually called a ‘word’, whether it 
represents a number or an instruction; ‘words’ representing numbers 
are normally used by being transferred between the store and the 
arithmetical unit, and ‘words’ representing instructions are normally 
used by being transferred to the control system. But since there is no 
distinction, within the machine, between numbers and instructions, it is 
possible to use the arithmetical unit to build up, transfer, and modify 
the operating instructions themselves as the calculation proceeds. 

The effect of this on the organization of a calculation is so profound 
that there may be httle relation between the processes of organizing a 
calculation for machines which do and for those which do not provide 
this possibility. However, its importance is now well realized, and it is 
provided on most of the machines already (1957) in operation and is 
likely to be provided on all future machines. It will be assumed in 
the rest of this chapter. 
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There are two main forms for instructions; these can be illustrated 
by an example. Suppose we want the machine to form the sum of the 
contents of storage locations and and to put the result into location 
ng. This could be done by a single instruction which could be written 

symboUcaUy (7(„j)+C(».,) to(12.1) 

An alternative is as follows. Suppose that the arithmetical unit is of a 
kind which includes as one of its components a register, usually called 
an ‘accumulator’, corresponding to the ‘product register’ or ‘accumu¬ 
lator’ of a desk machine, which accumulates the sum of numbers added 
into it until it is cleared; the content of the accumulator will be written 
C{Acc). Then the required operation can be done by the three separate 
instructions: 

(7(%) to Acc, G{n^ to Acc, C(Acc) to Wg. (12.2) 

Each of the instructions of the form (12.1) specifies three addresses in 
the store, whereas each instruction of the form (12.2) specifies a single 
such address. These forms of instruction are consequently known as 
the ‘three-address’ and ‘one-address’ forms respectively. 

An instruction specifies an operation to be carried out. But it is also 
necessary to specify the sequence in which such operations are to be 
carried out; that is to say, after carrying out one operation, the machine 
must be enabled to select the instruction for the next operation. In a 
machine in which instructions are contained in the same store as numbers 
this can be done in two ways. One is to include in each instruction the 
address from which the next instruction is to be taken; with a three- 
address specification of the operation to be carried out, this gives 
altogether a ‘four-address’ form of instruction. Another way is normally 
to store instructions at addresses numbered serially in the same order 
as the time-sequence in which they are to be carried out. This will be 
referred to as ‘serial storage’ of instructions. In this case the address 
of the instruction currently being carried out is recorded in a register 
whose content is normrlly increased by unity on the completion of this 
instruction, and the content of this register is used to control the selection 
of the next instruction. Then an explicit specification of the address of 
the next instruction is needed only when it is required to depart from 
the serial order in which the instructions are located in the store. 

The control system of a machine will depend on the standard form 
adopted for instructions and the means adopted for selecting the next 
instruction, and once the machine is built, the instructions used must 
conform to the type for which the control system has been designed. 
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These features, rather than the physical form of the store or of the arith¬ 
metical unit, are the essential features of a machine from the point of 
view of the user. 

12.2, Preparation of calculations for an automatic digital cal¬ 
culating machine 

The process of preparing a calculation for an automatic digital machine 
can be broken down into two parts, often called ‘programming’ and 
‘coding’. 

By the ‘program’ for a calculation is meant the schedule of operating 
instructions which has to be provided to the machine in order that it 
shall carry out the calculation. ‘Programming’ is the process of planning 
the sequence of operating instructions required, and ‘coding’ is the 
process of translating these instructions into the particular form in which 
they are supplied to the machine. In simple calculations these are 
hardly two distinct processes, but in more elaborate calculations it is 
convenient to treat them as separate. 

A process of programming is required in a hand calculation; before 
we can start doing any calculation we must decide just how we are 
going to do it. For work with an automatic machine, programming 
may involve breaking down the calculation to a sequence of the elemen¬ 
tary operations, such as addition, multiplication, and selection of the 
next instruction, which the machine can carry out. But the machine 
and the process of providing it with instructions may be such that groups 
of operations for standard processes, such as evaluation of cos a; given 

b 

the value of x, or of J f{x) dx given the values oif{x) at a set of values 

a 

of X, can be programmed and coded once for all. If this can be done, 
each such process can be regarded as a unit in programming a calculation, 
and not analysed into elementary operations. The program for such a 
standard process will be called a ‘sub-routine’. The possibility of using 
such sub-routines freely greatly lightens the work of preparing a calcula¬ 
tion for a machine, and the machine and the form of its instructions 
should be planned to provide this facility. The use of the same form 
in the machine for instructions and for numbers, and the freedom which 
this gives to modify instructions by arithmetical and other operations 
on them, are important features in making it easy to provide and exploit 
the possibilities of using such sub-routines. To a potential user of an 
automatic machine, means of organizing calculations for it are as impor¬ 
tant as the provision of the machine itself, and the provision of a library 
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of sub-routines for standard processes is an important step in this 
direction. 

Although the various kinds of machines differ considerably from one 
another in their internal organization and operation, the general process 
of programming a calculation will be much the same for any of them, 
for it depends primarily on the structure of the sequence of operating 
instructions required to carry out the calculation. Some characteristic 
features on an individual machine may, however, affect the details of the 
programming. Such features are 

(i) The standard form of operating instructions adopted; whether 
this is, for example, a one-address or four-address form. 

(ii) The facilities provided by the standard instructions; for example, 
whether division can be carried out directly or has to be done by 
means of an iterative process which has to be programmed. 

(iii) The criteria which it is possible to use for discrimination between 
possible alternative courses of procedure; for example, whether 
it is only possible to discriminate on the sign of a number or also 
on criteria such as the likeness or unlikeness of signs of two 
numbers. 

The process of coding does, however, depend on the particular 
machine. It does not depend primarily on the physical form of the store 
or on the way in which numbers are represented in the machine, but on 
two features, namely the standard form of instructions and the means 
of selecting the next instruction, as explained at the end of § 12.1. 

The details of programming and coding a calculation will be different 
for different machines, depending on the facilities provided by the 
machine itself and by the organization associated with it (such as the 
range of the available library of sub-routines).f 

12.3. Hand and automatic calculation 

Almost any method for doing a calculation by hand, that is, with a 
desk machine but without the use of an automatic machine, can be 

t A fairly full account of programming and coding for one machine, the EDSAC at the 
Mathematical Laboratory at the University of Cambridge in its original form, is given in 
M. V. Wilkes, D. J. Wheeler, and S. Gill, The Preparation of Programs for an Electronic 
Digital Calculating Machine (Addison-Wesley Press, Cambridge, Mass., 1951; a second 
edition is in course of preparation). See also A. D. and K. H. V. Booth, Automatic Digital 
Computers (Butterworth, 2nd edn., 1956), chapters 13-16; W. J. Eckert and R. James, 
Faster, faster (an account of the U.S. Naval Ordnance Research Computer NORC and 
methods of programming for it (McGraw-Hill, 1955)): M. V. Wilkes, Automatic Digital 
Computers (Methuen, 1956), ch. 3; J. H. Wilkinson, Phil, Trans. Roy. Soc, 248 (1955), 
243; and a number of shorter articles in M.T.A.C, 
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programmed for an automatic machine. A possible exception is the 
relaxation process (§§ 8.5, 10.61) for which it would be difficult to 
formalize the judgements one uses in practice about when and by how 
much to over-relax, and when to use group relaxations, and to express 
these judgements in terms of operating instructions to an automatic 
machine. But it does not follow that the best method for a hand cal¬ 
culation is the best method for an automatic machine. There are three 
main reasons for this difference. 

First, in most hand calculations of any magnitude, the time taken in 
carrying out the numerical work is substantially longer than the time 
taken in planning it, whereas with an automatic machine the time taken 
to carry it out may be shorter than the time taken to program and 
code it. Thus in a hand calculation it is worth spending some time in 
planning the calculation to save numerical work, whereas with an 
automatic machine it may be best to obtain the same results by a simple 
process involving a large number of steps to save the time that would be 
taken in plaiming, programming, and coding a less simple method using 
fewer numerical steps. For example, on an automatic machine a rela¬ 
tively large number of repetitions of a simple first-order iterative process 
(§ 9.3) may be preferable to a smaller number of repetitions of a more 
complicated second-order process. And in calculating an integral as a 
function of the upper limit, it might be best with an automatic machine 
to use a very simple integration formula, such as Simpson’s rule or even 
the trapezium rule, with a large number of short intervals, 0-01, or 0-005, 
or even perhaps 0-001, when in a hand calculation one might prefer to 
use an integration formula to sixth or eighth differences of the integrand, 
with interval 0-1. 

Secondly, the storage capacity of a machine is limited, whereas that 
of the working sheets of a hand calculation is practically unhmited. This 
has several reactions on programming for an automatic machine. For 
example: (i) use of many repetitions of a simple procedure which can be 
programmed in a few instructions is preferable to a few repetitions of a 
more elaborate procedure for which the longer program would take 
more storage space; (ii) a strictly repetitive procedure is to be preferred i 
to a procedure which is mainly repetitive but for which special occasional 
processes have to be used in addition: for example, in some processes | 
for the numerical integration of differential equations a special pro¬ 
cedure is needed for the first interval of the integration; the instructions 
for this special procedure will take up some storage space but will be used 1 
once only for each solution, and a method which does not require a special I 
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starting process may be preferred; (iii) it may be preferable to calculate 
values of standard functions, such as circular and exponential functions 
and their inverses, as they are required, rather than to store tables and 
the instructions for entering them and interpolating in them. 

And thirdly it is usually no shorter or easier to calculate with simple 

numbers than with numbers of many digits. For example, if e?> is cal- 

1 

culated from a series, then in calculating, say, J e^’ dx by a Gauss formula 

0 

(see § 6.61) the fact that e*’ is required for values of x such as 0*230765 
{x^ for a five-point Gauss formula, see § 6.61) and not only for simple 
values is no drawback. 

All these differences have considerable influence on the choice of 
methods for carrying out calculations by automatic machines. For 
example, for evaluating an integral between fixed hmits formulae of the 
Gauss type are much more attractive for work with an automatic 
machine than for a hand computation. Also for evaluating an integral 
as a function of its upper hmit it might even be better to do a number of 
independent integrations by means of a Gauss formula, with different 
values of the upper limit, rather than to build up the integral by accumu¬ 
lating a sequence of contributions to it. And for the solution of partial 
differential equations of elUptic type, a form of the Richardson-Lieb- 
mann process (§ 10.64) maybe more convenient for an automatic machine 
than the relaxation process. 

Some work has been done on the development of methods particularly 
suited to the capabiUties and hmitations of automatic machines, but 
the main developments of this branch of numerical analysis probably 
stUl lie in the future. 


EXAMPLES 


Note: Several of the following examples are specimens of types of which the reader 
can make up other examples for himself. For instance, in the first example 
could be replaced by some other number, and in Example 6, the series to be 
evaluated could be replaced by the series solution of some other second-order linear 
differential equation with the first derivative absent. 

1. Given = 1*105171 to six decimals: 

(i) Calculate up to n = 10 by successive multiplication and transfer. 

(ii) Check the results by verifying the following relations between the 2 /„’s: 


2/10/2/7 — 2/3* 
y^lVb ~ 2 / 4 * 
2/8/2/6 = 2/2* 


2/6 X 2/4 = 2/io» 

2/7 X 2/2 = 2 / 9 > 
2/5 X 2/3 = 2 / 8 » 


(iii) Check the results by differencing the values of (including the value 
2/0 = 1) to second differences, and verifying that (8^2/n)/2/n is constant. 
{Note: The main purpose of this example is to give practice in the use of a desk 
machine; the method of checking in section (ii) of the example is not recommended 
as a standard procedure for regular use.) 

2. Given = T133148, evaluate 2[(cosh 0*125) — 1] without writing down 
any intermediate results. 

Prove the relation = 2[(coshSa:)—l]e® and use it to build up up to 

n = 10. 

3. Given sin 10® = 0*1736482, find (1 —cos 10®) to seven decimals by an iterative 
process based on the formula 


1—cos:?: = sin2a:/[2 —(1 —cosx)]. 


Prove the relation S^(sina:) = — 2(1 —cos8:r)sinx, and use it to build up a table 
of 8in(n. 10®) as far as n = 9. 


4. Calculate 2 ^71/ X = 1*274, Xg = 0*984,^3 = T577, 

n In 

= 0*126 without writing down any intermediate results. 

5. Find 3 X (£3. 85 . 7d,) +19 X (£1.17^. 9d.)+16 X (£2. 25, lid.) using an ordinary 
desk machine, setting the sterling amounts in £. 5 . d. and exhibiting the results in 
£. 5 , d., without writing down any intermediate results. 

{Note: Assign the three right-hand places on the setting levers or keyboard to 
pence, the next three to shillings, and the rest to pounds. After forming the sum, 
reduce the number of pence by a multiple of 12, and add that same multiple to the 
shillings, by adding 988 in the right-hand three places until the number of pence 
in the result is less than 12. Treat the shillings similarly.)f 



to five decimals for x = 0(0*2)2*0, keeping seven decimals in the individual terms 
and rounding off the sums to five decimals. 


f This procedure for using a decimal machine for certain calculations in sterling was 
shown to me by Dr. L. J. Comrie. 
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Check the results by evaluating from the differential equation y'* = \-\-xy 
satisfied by this function y, and verifying the relation 

8^2/i = (8x)2[2/;+*822/r-^8X] + 0(8a:)s. 

7 . Show that the function y = (sinx—a:cosa;)/x satisfies the differential equa¬ 
tion y" -\-{l^2lx^)y = 0. 

Evaluate this function to five decimals, for x = 0(0* 1)2*2, from its power series 
expansion, and check the results by use of the differential equation. 

8 . It is given that if Z(a, 8) is the common limit of the sequences {a„}, {8^} given by 


^n-fl — V(®n^n) ^ 


thenf 


(Zq — (^9 
8o = hf 

J d0/(a2cos^0+8^sin20)l = 7r/2Z(a,8). 


(A) 


It is also given that the complete elliptic integrals 

\tt in 

K{k) = J (Z0/(1-Jfc2sin20)i, E{k) = J {\-k^s\rv^e)i dO 
0 0 

satisfy the differential equations 

dKjdk = [E-{\-k^)K]lk{\-k^), dEjdk = {E-K)lk. 

Use the result (A) to evaluate {2l7T)K(k) to five decimals for k"^ = 0(0* 1)0*8. 
Show from equations (B) that K(k)y as a function of satisfies the equation 


(B) 




d(P) 


and use this equation to check the values oi K{k) derived by use of formula (A). 


9. Build up the cubic f{x) — x^—’5x^-\-6x-{~l between x = 2 and x = 3 at 
intervals of 0*1 by means of a difference table. From these results estimate the 
position Xj^ and magnitude of the minimum of/(a;) near x = 2*5. Verify by solving 
the quadratic for and evaluating/(a;^). 

{Note: Change to ^ = x —2 as variable, verifying the transformed form of the 
cubic by evaluating it for ^ = 0, ±2 and comparing with the values of f{x) 

on p. 42.) 


10. Evaluate 0*623a;^—l*876a:^ + 5*623a;-|'2*875 to three decimals for 


X = 0(0*32)2*56 

and check the results by differencing. 

11 . The following values are alleged to be copied from a table of x^. Locate and 
correct the mistakes by examination of the differences. 


X 

f(^) 

X 

fix) 

27 

300000 

35 

3-27107 

28 

•03659 

36 

•30193 

29 

•07232 

37 

•33332 

30 

•10723 

38 

•36198 

31 

•14318 

39 

•39121 

32 

•17480 

40 

•41995 

33 

•20753 

41 

•44852 

34 

3-23961 

42 

3-47603 


t See E. T. Whittaker and G. N. Watson, Modern Analysis, ch. 22, example 46. 
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12 . Using 6-figur6 tables of sina;°, calculate the function 

y - ,inx“- 2 .I 0 -[“J^^^]cxp[-(x- 60 )V 100 ] 

for X = 30(1)70, and round off to five decimals. 

Compare the second and fourth differences of the rounded values of y with those 
of five-figure values of sinx°. 

Repeat for x = 25(5)75 and for x = --20(10)120. 

(Note: t/—sinx® can be regarded as an ‘error’ in a table of sinx®. The purpose of 
this example is to illustrate that smooth differences do not necessarily imply free¬ 
dom from error; the differences of y at intervals 1 in x are no more irregular than 
those of sinx. It also shows that a table may appear smooth on a small scale as 
represented by the differences at a small interval of x, but unsmooth on a large 
scale.) 

J-\ 

13. Show that the sum 2 alternate second differences, can be expressed 

i = o 

in terms of the operator U — (8x)d/dx as (tanh iU)(/ 2 j—/o). 

Deduce the value of this sum when / is a periodic function which is an even 
function both of (x—X q) and of (x — x,^j)y and examine whether or not this result 
is independent of rounding errors in the / values. 

14. From the table of the function y calculated in Example 6: 

(i) Use the‘half-way’ interpolation formula to obtainy atx = 0-7, 0*9, 1*1,1*3. 

(ii) Interpolate y for x = 0-95(0-01)l*00 by Everett’s formula. 

(iii) Find the value of x for which y = 0*5 

(а) by inverse interpolation using the values at Od intervals only; 

(б) by inverse interpolation using the values at 0*01 intervals calculated 
under (ii). 

15. From a table of sinx at intervals of 10° in x (see p. 60): 

(i) Find sin 23° 20' and sin 26° 40'. 

(ii) Find sin"^ 0-40 

(a) by inverse interpolation using a formula involving the differences of 
sinx as a function of x; 

(b) by using Lagrange’s interpolation formula, treating x as a function of 
sinx, and verifying by interpolating in the table of sinx for the value 
of sin“^ 0*40 obtained; 

(c) by using the divided differences of x os a function of sinx. 

16. Construct a table of values of log(n!) to five decimals for n = 5(1)12. Use 
this table to interpolate log(x!) for x = 8^ and 9^, and verify that the interpolated 
values satisfy the relation (9J)!/(8J)! = 9J. 

Derive a value for (i)! = and hence a value of tt. 

17. Given the values 

X = 0 1 2 3 4 5 

2 / = 0 1 8 27 64 125 

examine the result of attempting to interpolate x for y = 20by a six-point Lagrange 
formula for x as a function of y. 

K the calculation were done by the use of divided differences what symptoms 
would suggest that the result should be accepted with suspicion ? 

U 


0018 
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18. Continue to x = 0-285 the sub tabulation started in the example in § 5.61. 

X 

19. Continue to x = 1-6, by intervals x = 0-05, the evaluation of 
started in the example in § 6.4. 

20. Show that 

jf{x)dx = 4(8x)[l ++ 9f8S*+5-6^‘oS®]/o+^(^^)“- 

*-i 

21. The function erfcx is defined by 

00 

erfcx = (2/7x1) Je-o'du;. 

X 

Evaluate erfcx for x = 0(0-1)1-2 to five decimals by quadrature. Evaluate 

00 

2 J evfcwdw for x = 0(0* 1)1*0 by quadrature and check by use of the relation 

X 

2 j erfetv dw = ( 2 / 7 r^)e“®*—2a;erfca:. 

X 

00 

{Notes-, (i) erfeO = 1; 2 J erfcti; dw = 2/7t* = 1-128379; (ii) the relation between 

0 

00 

J erfety dw and erfcx is obtained by integrating by parts.) 

X 

00 00 

22. Evaluate ^ [e-^^/{x+l)] dx and ^ [e-’^^/(x+2)] dx to five decimals by 

0 0 

00 

quadrature. Check by evaluating the difference J [c~^^l{x-\-l){x+2)] dx between 

0 

these integrals by an independent quadrature. 

IT 

23. Evaluate y = (I/tt) J cos(xsin0) d9, to four decimals, by quadrature for 

0 

X = 0(j7r)f7T. Estimate, as closely as you can from the results, the smallest 
positive value of x for which y = 0. 

24. Continue to x = 1*6 the integration of the equation y" = (l~x^)y started 
in the worked example in § 7.2. 

25. Continue to x = 2*0 the integration of the equation y' =x 1 ~2xy started in 

X 

the example in § 7.3. The solution of this equation is y = 6““^* J e^^dw. Compare 

0 

the results of the integration of this differential equation with those of the worked 
example in § 6.4 and its continuation in Example 19. 

00 

26. The function /(x) = J e~^^l(x-\-u) du satisfies the differential equation 

0 

/'+2x/= --+ 77 ^ (see § 6.56). 

X 

Starting from the value of/(l) obtained in Example 22, integrate this equation as 
far as X = 2. Compare the value of/(2) obtained by integration with the value 
of /(2) obtained in Example 22. 
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27. Evaluate y *= ( 8 / 77 ) j* to four decimals for x = 0(J)2 

0 

(а) by expanding the integrand in series, integrating term by term with respect 
to 6y and evaluating the resulting series in x; 

(б) by quadrature; 

(c) by obtaining a seeond-ordcr differential equation satisfied by y as a function 
of X, and evaluating the appropriate solution by numerical integration. 

28. Find, to two deeimals, the solution of the equations 

18X-42/ + 32 = 53, 

10a:4-162/ + 23 = 87, 

5x+3i/+92 = 21 

(a) by elimination; (6) by relaxation. 

29. A eubic y = + + takes the values y = 12, 6, 0, 12 for 

X = —2, 0, 1, 3 respeetively. Find the values of the coefficients Oq, Oj, cl^ 

(i) by substituting x = —2, 0, 1, 3 and solving the resulting simultaneous 
equations for the coefficients; 

(ii) by use of divided differences (see § 5.72). 

30. Use Milne’s method (§ 6.8) to obtain an expression for the error term of 
the integration formula 

\f(x)dx = i(Sx)[/o+/i-i(Sx)(A'-/i)]. 

31. (i) Invert the matrix 

( 5 7 6 5\ 

7 10 8 7 \ 

6 8 10 9 r 

5 7 9 10/ 

(ii) Find the eharaeteristie values and charaeteristie veetors of this matrix. 

32. Construet the inverse of the matrix 

/-23 11 1\ 

I 11 -3 2 

\ 1 2 1/ 

from its eharaeteristie veetors and the reciprocals of its charaeteristie values as 
determined in § 8.72. 

{Note: This is the matrix whose inverse is found by elimination in the worked 
example in § 8.3.) 

33. Solve the equations 

14xj ^^2 H” i ^^3 “h ^ 134, 

7xj-j-11^2 “{“ 1 8 x 3 -f-= 70, 

17xi+ 13 x 2 + 42 x 3 -llx^ = 77, 

8 xj +4x2— 11 x 3 + 30 x 4 = 70. 
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34. Find the solution of the equation y" = x^y— 1 for which i/ = Oata; = ±2 

(i) by evaluating a particular integral and a complementary function by step- 
by-step integration and forming the appropriate linear combination; 

(ii) by a relaxation method. 

35. Work out a relaxation method for finding a solution of = 0 for a system 
with symmetry about an axi<s. Apply it to find an approximate solution of = 0 
for the axially-symmetrical system formed by rotating Fig. 20 (p. 246) about its 
axis of symmetry. 

36. V^F = 0 in the space between the planes z = 0, a. On the plane z = 0, 
F = 0 for r > a and F = J^^iTcr) for r < a, where ka is the first root of Jq(x) = 0; 
on the plane z = 2a, F = 0. Find to two decimals the variation of F in the space 
between the planes. 

37. F satisfies the equation V^F = 2/a^ inside a square of side a, and F = 0 on 
the boundary of the square. Find to three decimals the value of F at the centre 
of the square. 

38. F satisfies the equation V^F = —320/a^ within a circular cylinder of radius a 
and length 2a, and F = 0 on the bounding surface. If {R,z) are radial and axial 
coordinates in a section containing the axis of the cylinder, with origin at its centre, 
obtain the finite difference equations for F on a grid of square mesh with 
hR = 8z = a/n (n integral), and solve them to two decimals for n = 2 and 4. Use 
the results to estimate, by Richardson’s process (§ 7.51), the value of Fat the centre 
of the cylinder for the solution of the differential equation. 

{Note: For n = 2, use a direct method for the solution of the finite-difference 
equations; for n = 4, use a relaxation procedure.) 

39. Find to three decimals the coefficients in the quadratic factors of 

/(z) ^ z^-4‘0z3 + 7*8z2 —8-2Z + 5-6. 

Hence obtain to three decimals the roots of f{x) = 0. 

40. Find to two decimals the other roots of the simultaneous non-linear equations 
of which one root is found in § 9.6. 

41. Use the iterative formulae 

(«) 2/n+l = i[2/n + («/j/n)]. (i>) Vn+l = Vni^-VDl^a 

for c* to evaluate V6 and V60. 

42. Show that the formula 

J/n+l = VnlP+^-ay^VP 

gives a second-order iterative process for 1/aVP. 

43. Devise a second-order iterative method on the lines of Example 42 to find 
(l/9a)^/®. Use it to find (8/9)V® and (1/9)V® to eight decimals; check by verifying 
that the ratio of the results is 2V*. 
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INDEX 


Accumulator, of desk machine, 11, 15, 48, 
60; of automatic digital machine, 282. 

Accuracy, 9, 190, 273. 

Adding machine, 11, 19, 42. 

Addition, on desk machine, 12, 14. 

— of complex numbers, 235. 

Airy functions, 77, 128, 264. 

Aitken, a. C., 83. 

Algebraic equations: linear simultaneous, 
166-209; non-linear, in one variable, 
210-27 ; non-linear, in two or more vari¬ 
ables, 228-34. 

Analysis of observations, 4, 268. 

Automatic digital calculating machines, 
25, 279-86; accumulator of, 282; arith¬ 
metical unit of, 282; control in, 280, 282 ; 
instruction in, 281, 282; program for, 
281; serial storage in, 282 ; store of, 280 ; 
sub-routine for, 280, 283; word in, 281. 

Automatic division, on desk machine, 19. 

Auxiliary function in smoothing, 276. 

— variables in tables, 22. 

Averaging operator /x, 38, 52; inverse of, 
62. 

Back-substitution, 175, 178. 

Bernoulli numbers, 101. 

Bessel fimctions, 23, 27, 190, 271. 

— interpolation coefficients, 68, 69, 95. 

Bessel’s interpolation formula, 64, 67-73, 

86, 90, 91, 95, 98, 117. 

Bickxey, W. G., 55, 91, 199, 237. 

Boundary conditions, 134, 155, 240, 243. 

Calculating machines: automatic, 279-86; 
desk, 11-20, 27. 

Cardinal function, 93-96, 117. 

Characteristic values: of matrices, 169, 
171, 196-209; of ordinary differential 
equations, 162-5, 209; of partial differen¬ 
tial equations, 252. 

— vectors of matrices, 196-208. 

Characteristics, of hyperbolic partial differ¬ 
ential equations, 257-63. 

Checking and checks, 3, 4, 17, 28, 29, 30, 
39, 42, 43, 44, 47, 62, 74, 81, 82, 83, 93, 
104, 105, 106, 108, 136, 137, 138, 140, 
151, 160, 173, 174, 184, 187, 189, 201, 
205, 222, 230, 236, 265, 267, 274. 

Choleski method, for linear simultaneous 
algebraic equations, 180; for matrix in¬ 
version, 185; Thomas-Fox application 
to differential equations, 191. 


Circular functions, 32, 34, 60, 62, 78, 286. 

Clearing, 11. 

Complement, 16. 

Complex numbers, 235, 236. 

— variable, functions of, 235, 239. 

Comrie, L. J., 64, 68, 70, 75, 76, 79, 82, 83, 

91. 

Continued product, 27. 

Convergence, 3, 189. 

Critical tables, 21. 

Cross-sum check, 173, 174, 176, 177, 179. 

Cubic equation, 220. 

Cylindrical coordinates, finite differences 
in, 240. 

Dedekind section, 3. 

Deferred approach to the limit, 151. 

Derivatives and differences, 49, 50, 55-58, 
126, 127. 

Determinant, 166, 168, 169, 177, 197. 

Difference operators, 36, 50-58; inverses 
of 52, 62. 

Differences, finite, 35-58; building up from, 
41, 48; checking by, 39, 43; and deriva¬ 
tives, 37, 49, 55-58, 191, 193; effects of 
errors on, 36, 43-46; in terms of function 
values, 38, 53; notation for, 37; smooth¬ 
ing by, 277. 

Differentiation: formulae, 55, 126-7; gra¬ 
phical, 129; numerical, 124-8. 

Diffusion equation, 242, 254-7. 

Digital machine, 279. 

Direct interpolation, 59. 

Divided differences, 86-89. 

Division, on desk machine, 17. 

— of complex numbers, 235. 

— of a polynomial by a quadratic, 222. 

Doodson, a. T., 276. 

Elimination, 167, 173-9, 228. 

Elliptic partial differential equations, 190, 
242, 243, 244-53. 

End-figure method, for subtabulation, 79. 

Errors: see Rounding errors. Systematic 
errors. Truncation errors. Mistakes. 

Euler-Maclaurin formula, 101, 104, 115, 
116, 117, 266, 269. 

Euler’s transformation of a power series, 
265. 

Everett interpolation coefficients, 67, 70. 

Everett’s interpolation formulae, 64, 66, 
70-74, 75, 80, 86, 98. 

Exponential extrapolation,’30,' 90, 216. 

— function, 27, 76, 286. ^ 
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INDEX 


Factorial polynomials, 40, 65, 66, 67, 68. 

Factorization: of differential equation, 161; 
of matrix into triangular matrices, 180, 
192. 

Ferrak, W. L., 93, 95. 

Finite differences: in cylindrical coordi¬ 
nates, 240; in two dimensions, 236; see 
also Differences, finite. 

Formulae; evaluation of, 4, 26-32; sig¬ 
nificance of, in numerical work, 26. 

Forward differences, 37, 51, 114, 265; in 
integration, 114; in interpolation, 63. 

Fox, L., 147, 154, 190, 191, 196, 245. 

Frequency analysis, 4. 

Function, continuous, 33. 

— of a complex variable, 235, 239. 

— of two variables, 235-63. 

Gamma fimction, 33. 

Gauss integration formulae, 120-4, 286. 

Gibbs phenomenon, 268. 

Goodwin, E. T., 117, 119, 147, 154. 

Graphical methods, 129, 210, 228, 230-3, 
276. 

Gregory integration formula, 114. 

Group relaxations, 188. 

Guarding figures, 6, 174; in interpolation, 
71, 72, 81. 

/t*-extrapolation, 148, 152, 254, 255, 274. 

Half-way interpolation, 61, 108. 

Harmonic analysis, 4, 94, 268. 

Heat-conduction equation, 242, 254-7. 

Hollerith machines, 25. 

Hyperbolic functions, 27. 

— partial differential equations, 243, 
253-8. 

Ill-conditioned equations, 168, 177, 179, 
190, 206. 

Indeterminate forms, 30. 

Initial conditions, 24, 243. 

Input, in automatic machine, 280. 

Integral; between fixed limits, 97, 113-24 ; 
by solution of a differential equation, 
119; as function of upper limit, 97, 
104-9. 

— condition, on solution of differential 
equation, 135, 165. 

— equation, 4. 

— parametric, 118. 

— twofold, 112. 

Integrating factor, 146. 

Integration formula, 57, 58, 98-104. 

Integration, numerical, 4 ; of a given func¬ 
tion of a:, 97-123; of an ordinary 
differential equation, 97, 134-65, 191-6, 
of a partial differential equation, 242-63. 


Interpolation, 4, 22, 59-96; linear, 69, 60; 
non-linear, 23, 69, 61-96; in complex 
plane, 240. 

Interval length, change of: in quadrature, 
108; in integration of differential equa¬ 
tion, 139, 249. 

Inverse interpolation, 69, 60, 70, 75, 89, 
210, 215, 229. 

— of a matrix, 7, 168, 178, 185, 191, 199. 

— operators, 52, 55, 62. 

Iterative process, 211, 285, 292; for alge¬ 
braic equation, 211; for characteristic 
values of a matrix, 199; for differential 
equations, 153, 157, 193; for inverse 
interpolation, 90, 214; for quadratic 
factor of polynomial, 224. 

Jury problem, 135. 

Lagrange integration formula, 114. 

— interpolation coefficients, 74. 

— interpolation formula ; equal intervals 
of argument, 74^76; for inverse inter¬ 
polation, 91—93; xmequal intervals of 
argument, 82, 86, 114, 122. 

Laplace’s equation, 235, 239, 246, 248, 250, 
253, 269. 

Laplacian operator; in two dimensions, 
235, 238; in three dimensions, 240. 

Latent roots (of matrices), see Characteris¬ 
tic values. 

— vectors (of matrices), see Characteristic 
vectors. 

Leading differences, 37. 

Legendre polynomials, 120, 271* 

Liebmann, H., 253. 

Liebmann’s process for Laplace’s equation, 
253. 

Limiting process, 3. 

Linear cross-mean, 84. 

— differential equations; see Ordinary, 
Partial differential equations. 

— interpolation, 59, 60. 

— operators, 52, 93, 130. 

— simultaneous equations, see Simulta¬ 
neous equations. 

Lower triangular matrix, 168, 180, 185, 
192. 

Madelung, E., 154. 

Madelung transformation, 154. 

Marching problem, 134. 

Matching process, in solution of linear 
differential equation with two-point 
boundary conditions, 160, 163, 164. 

Matrices, 4, 168, 171, 180; characteristic 
values of, 171, 196-209; inverse and in^ 
version of, 168, 178, 185, 191, 199. 
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Mean differences, 38. 

Michel, J. G. L., 64. 

Miller, J. C. P., 47. 

Milne, W. E., 93, 129. 

Mistakes, 5, 7, 27, 44, 45, 105, 174, 215. 

Modified differences, 70, 75. 

Morris, J., 170. 

Multiple roots of algebraic equations, 217, 

222 . 

Multiplication, on desk machine, 12, 16. 

— of complex numbers, 235. 

‘National’ calculating machine, 24, 77. 

Neighbouring roots of algebraic equation, 
217, 218. 

Newton’s forward-difference interpolation 
formula, 63. 

— iterative process for square root, 213. 

Newton-Raphson iterative process, 214, 

215. 

Nominal accuracy, see Precision. 

Non-linear interpolation, 59, 61—91. 

Normal equations, 171. 

Normalization of characteristic vectors, 
197, 199. 

— condition on solution of a differential 
equation, 165. 

Numerov, B., 142. 

Olver, F. W. j., 142. 

Operations table, in relaxation calcula¬ 
tion, 186, 189, 208. 

Order of an iterative process, 212. 

Ordinary differential equations, 2, 24, 119, 
134-65, 191, 255, 285; boundary con¬ 
ditions for, 134, 155; first-order, 143, 
146; linear, 23, 135, 142, 148, 154, 155; 
second-order, with first derivative ab¬ 
sent, 135-43, 191, with first derivative 
present, 148, with two-point boundary 
conditions, 159-65; second-order linear, 
relation to simultaneous algebraic equa¬ 
tions, 191-6; third and higher orders, 149. 

Orthogonality of characteristic vectors of 
a symmetrical matrix, 197, 200, 201, 
205, 206; of Legendre polynomials, 121. 

Parabolic partial differential equations, 
243, 253-7, 258. 

Partial differential equations, 242; boun¬ 
dary conditions for, 243; elliptic, 190, 
243, 244-53, 258; hyperbolic, 243, 257- 
62; parabolic, 243, 253-7, 258. 

— fractions, 83. 

Pivotal coefficient, 174, 177. 

— equation, 173, 175, 177. 

— value, 77, 78, 79, 80. 

Poisson’s equation, 242, 244, 248, 249. 


Polar coordinates, 235, 240. 

Polynomial; differences of, 39; divided 
differences of, 87; derivatives of, 88; 
evaluation of, 28, 41; factorial, 40. 

'— equation, 218, 221. 

Power series, 29, 32, 264. 

Powers-Samas machines, 25. 

Precision, 9. 

Processes, numerical, 1, 3, 26. 

Proportional parts, in interpolation, 61. 

Punched-card machines, 25. 

Purification process, for characteristic vec¬ 
tors of matrices, 201. 

Quadratic equation, 32, 219. 

— factor of polynominal, 224. 

Quadrature, 97-124. 

Quartic equation, 220. 

Rayleigh’s principle, 199. 

Recurrence relation, 27, 190, 271. 

Reduced derivatives, 76, 150. 

Relaxation method; for algebraic equa¬ 
tions, 185—91, 257, 285 ; for characteristic 
values of matrices, 207; for ordinary 
differential equations with two-point 
boundary conditions, 196; for elliptic 
partial differential equations, 245-53. 

Residuals, 167, 171, 177, 179, 186, 187, 
188, 245, 248. 

Resistance function, 275. 

Riccati transformation, 155, 201. 

Richardson, L. F., 134, 152, 201, 253. 

Richardson-Liebmann process, 253, 256. 

Root-squaring process, 221. 

Rounding errors, 5, 45, 46, 106, 141, 177, 
179, 217, 218, 258, 272; see also Toler¬ 
ance. 

Round-off, 5, 42, 43. 

Rule of false position, 214. 

Second difference, 35, 127; direct evalua¬ 
tion, from function values, 47; building 
up from, 48, 79, 80, 136; see also Differ¬ 
ences, finite. 

Second-order iterative processes, 212, 216, 
226. 

Separation of variables, 242. 

Series, evaluation of, 29, 32, 265. 

Shift operators E^^, 51, 69, 265. 

Short-cutting, 17, 18, 

Simpson’s rule, 50, 102, 106, 115, 240, 285; 
correction to, 102, 240. 

Simultaneous algebraic equations; linear, 
4, 166-94, 244, 257; non-linear, 4, 228- 
34. 

Singularity, 22, 110, 118, 241; integration 
in neighbourhood of, 110, 118. 
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Slide-rule, 11, 23, 28. 

Smoothing, 4, 39, 272-8. 

Smoothness, 34, 47, 272. 

Southwell, R. V., 185, 188, 237, 251. 
Spencer, J., 275, 

Square root, 213 ; of complex numbers, 236. 
Statistics, 3, 4. 

Staton, J., 119. 

Sub-tabulation, 4, 70, 77-82. 

Subtraction, on desk machine, 12, 14. 
Summation of series, see Series. 

— operator cr, 52, 104, 277. 

Symmetrical matrix, 170, 197-201. 
Systematic error, 43, 44, 78. 

Tables, mathematical, 11, 20, 69, 74. 

— critical, 21. 

Tabulation, 1,4; use of auxiliary variables 
in, 22. 

Taylor series, 49, 51, 53, 63, 76, 149; in 
interpolation, 76; in integration of dif¬ 
ferential equations, 149. 

Temple, G., 189, 199. 

Terminal conditions, 243. 

Throw-back, 70, 71, 72, 75, 90. 

Tolerance, for rounding errors, 5, 6, 106, 
113, 128, 205, 219. 


Total, on adding machine, 20. 

Transfer, on desk machine, 15; on auto¬ 
matic machine, 280. 

Trapezium rule, for integration, 99, 103. 

Triangular matrix, 168, 175, 180-5, 192-3. 

Truncation errors, 5, 93, 129, 141, 244, 255, 
274. 

Two dimensions; finite differences in, 236— 
42 ; Laplace’s equation in, 235, 239, 244, 
245-51, 253. 

Two-point boundary conditions in differen¬ 
tial equations, 34, 155-65, 191, 255, 273; 
in linear equations, 159-65, 191-6. 

Turing, A. M., 169. 

Upper triangular matrix, 168, 175, 180, 
192. 

Wave equation, 242. 

Weber functions, 271. 

Weddle’s rule, for quadrature, 103, 115. 

Whittaker, E. T., 93. 

Whittaker functions, 60, 271. 

Wilkes, M. V., 180. 

Wilson, T. S., 170. 

Working sheet, 8, 25, 26, 275, 279, 280. 
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